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1.0  EXECUTIVE  SUMMARY 


1.1  OBJECTIVES  OF  THE  DEMONSTRATION 

The  use  of  military  vehicles  during  training  results  in  soil  disturbance  and  vegetation  loss,  with 
subsequent  increases  in  soil  erosion  rates,  sedimentation  in  streams,  habitat  degradation,  and 
numerous  other  secondary  effects.  The  National  Environmental  Policy  Act  (NEPA)  requires 
federal  agencies  to  evaluate  the  implications  of  their  plans,  policies,  programs,  and  projects. 
However,  accurate  assessment  of  military  training  impacts  is  limited  by  the  technical  data 
available  to  support  the  assessments.  This  project  demonstrated  the  use  of  the  Vehicle  Dynamics 
Monitoring  and  Tracking  System  (VDMTS)  to  assess  and  predict  military  vehicle  maneuver 
training  impacts  for  use  in  land  management  decision  making  and  NEPA  documentation.  The 
objective  of  this  project  was  to  demonstrate  and  validate  the  VDMTS  system  and  its  components 
through  a  series  of  controlled  field  studies  and  live  tracking  events.  A  controlled  field  study  was 
used  to  demonstrate  and  validate  that  the  hardware  can  sufficiently  characterize  vehicle  dynamic 
properties  (turning  radius  and  velocity)  to  accurately  predict  site  impacts  (area  impacted, 
vegetation  loss,  and  rut  depth  [RD]).  A  controlled  field  study  was  used  to  demonstrate  and 
validate  the  accuracy  of  VDMTS  impact  models  in  predicting  area  impacted,  vegetation  loss,  and 
RD  for  a  range  of  vehicles.  Field  studies  tracking  live  training  exercises  and  subsequent  field 
measurements  were  used  to  demonstrate  and  validate  the  VDMTS  hardware  and  model 
performance  in  predicting  site  impacts. 

1.2  TECHNOLOGY  DESCRIPTION 

The  VDMTS  approach  is  composed  of  three  components:  1)  vehicle  impact  models,  2)  vehicle 
tracking  hardware  and  software,  and  3)  vehicle  tracking  data  analysis.  The  approach  spatially 
characterizes  short-tenn,  direct  impacts  by  monitoring  individual  vehicle  locations  and  operating 
characteristics.  These  dynamic  characteristics  are  used  to  predict  area  impacted,  vegetation  loss, 
and  RD  based  on  vehicle  type  and  location.  Results  are  summarized  to  characterize  training  land 
use  patterns  and  quantify  the  severity  of  the  training  impacts. 

The  process-based  impact  models  predict  terrain  impacts  caused  by  wheeled  and  tracked  vehicles 
in  terms  of  percent  vegetation  cover  loss  (impact  severity  [IS]),  disturbed  width  (DW),  and  RD. 
Impact  models  predict  site  impacts  based  on  vehicle  static  (i.e.,  vehicle  weight  and  type),  vehicle 
dynamic  properties,  and  soil  properties  (soil  strength).  The  process  uses  vehicle  tracking  systems 
to  determine  vehicle  location  and  dynamic  operating  characteristics  (i.e.,  turning  radius  and 
velocity).  The  VDMTS  hardware  consists  of  a  Global  Positioning  System  (GPS)  receiver 
integrated  with  low-cost  inertial  sensors.  These  sensors  enable  measurement  of  vehicle 
kinematics,  dynamics  and  other  parameters  of  interest  that  enable  accurate  modeling  of 
environmental  impact.  The  system  thereby  provides  vehicle  dynamics  data  and  positional 
information  at  all  times,  even  when  GPS  is  unavailable. 

The  vehicle  tracking  data  are  analyzed  and  summarized  into  formats  appropriate  for  land 
management  decisions.  Analysis  routines  include:  1)  identification  of  individual  and  unit 
tracking  patterns,  2)  identification  of  on-  and  off-road  use  patterns,  3)  identification  of  existing 
and  emerging  trail  networks,  4)  vegetation  loss  estimates,  5)  identification  and  prioritization  of 
Land  Repair  and  Maintenance  (LRAM)  sites,  and  6)  development  of  data  for  carrying  capacity 
models. 
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1.3  DEMONSTRATION  RESULTS 


This  demonstration/validation  project  tested  and  validated  each  aspect  of  the  VDTMS  process  at 
multiple  levels,  specifically,  accuracy  of  the  hardware  and  models  in  combination,  durability  of 
the  hardware  under  multiple  training  events,  ease  of  use  of  the  VDTMS  process,  and  ability  to 
make  land-use  decisions  based  on  the  VDMTS  collected  and  summarized  data.  The  following 
quantitative  metrics  were  tested  to  assess  each  aspect  of  VDMTS  perfonnance:  1)  accurate 
VDMTS  hardware  measurement  of  vehicle  dynamic  properties,  2)  accurate  VDMTS  impact 
model  predictions  of  site  impacts  under  controlled  conditions,  3)  accurate  VDMTS  hardware 
measurement  of  vehicle  static  and  dynamic  properties,  4)  accurate  VDMTS  model  predictions  of 
site  impacts  during  live  training,  5)  VDMTS  hardware  durability  (in  single  live  training  event), 
6)  VDMTS  hardware  durability  over  14  live  training  events,  7)  ease  of  system  use,  and  8)  quality 
and  accuracy  of  data  for  land-use  decisions. 

Of  the  hardware  performance  metrics  cited  in  the  above  paragraph,  1,  3,  and  5-8  were  met. 
Metrics  2  and  4,  accurate  VDTMS  impact  model  predictions  in  controlled  and  live  events,  did 
not  meet  the  success  criteria  initially  proposed.  The  demonstrated  average  error  for  DW  was  14.9 
centimeters  (cm),  and  the  average  error  for  vegetation  removal  was  -1.8%.  These  results  are 
comparable  with  existing  site  and  vehicle-specific  empirical  model  predictions,  thus  reducing  the 
need  to  develop  models  for  each  site.  This  validates  the  use  of  the  theoretical  models  for  impact 
prediction. 

The  system  met  most  of  the  metrics  established.  While  it  failed  to  meet  some  metrics,  it  still 
perfonned  as  well  as  previous  methods  in  characterizing  vehicle  impacts,  reducing  the  relative 
cost  and  time  required.  Project  success  was  indicated  by  the  use  of  data  obtained  from  the  system 
by  the  installation  hosts  as  well  as  their  quick  implementation  of  the  technology.  Through  the 
course  of  the  project,  installation  Integrated  Training  Area  Management  (IT AM),  Environmental, 
Directorate  of  Public  Works  (DPW),  and  training  groups  have  used  results  obtained  from  this 
study.  Data  collected  were  used  in  land  management  and  vehicle  mobility  and  power  models. 
Study  results  also  informed  training  and  regulating  decisions. 

1.4  IMPLEMENTATION  ISSUES 

Implementation  of  the  VDMTS  approach  will  generally  be  driven  by  installation  land  manager 
requirements.  Despite  installation  acceptance  of  the  process,  the  main  implementation  issue  may 
be  caused  by  turnover  in  the  IT  AM  and  Environmental  installation  branches.  A  lack  of 
continuity  in  programs  from  this  turnover  may  result  in  issues  incorporating  the  system  into  that 
program.  An  additional  option  for  implementation  involves  using  existing  military  standard 
systems  (the  Army’s  Blue  Force  Tracking  [BFT]  and  the  National  Guard’s  Deployable  Force-on- 
Force  Instrumented  Range  System  [DFIRST]),  which  obtain  vehicle  location  and  time  data  on 
live  training  events  for  post-event  analysis.  This  option  would  decrease  costs  by  reducing  the 
time  and  equipment  required  to  obtain  vehicle  tracking  data.  However,  this  process  is  still  being 
developed  and  is  not  available  widely  at  this  time.  The  VDMTS  technology  has  been  and  will 
continue  to  be  valuable  in  obtaining  data  to  estimate  impacts  from  military  training.  This  Cost 
and  Performance  Report  assists  installations  in  making  informed  decisions  regarding  technology 
implementation  and  associated  costs. 
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2.0  INTRODUCTION 


2.1  BACKGROUND 

The  use  of  military  vehicles  during  training  results  in  soil  disturbance  and  vegetation  loss,  with 
subsequent  increases  in  soil  erosion  rates,  sedimentation  in  streams,  habitat  degradation,  and 
numerous  other  secondary  effects.  The  capacity  of  installation  lands  to  support  training  activities 
is  a  function  of  the  sensitivity  of  lands  to  specific  activities,  the  natural  recovery  rates  of 
vegetation,  the  weapon  system  characteristics,  the  doctrine  that  establishes  how  these  systems  are 
used,  and  the  actual  locations  where  activities  are  conducted.  Accurate  assessment  of  these 
impacts  is  limited  by  the  technical  data  available  to  support  the  assessments.  The  VDMTS 
consists  of  three  components:  1)  vehicle  impact  models,  2)  vehicle  tracking  hardware  and 
software,  and  3)  vehicle  tracking  data  analysis.  The  vehicle  impact  models  are  theoretical, 
process-based  vehicle  impact  models  used  to  predict  site  impacts  in  terms  of  disturbed  area, 
vegetation  loss,  and  RD.  Data  collected  by  the  VDMTS  hardware  are  used  with  the  impact 
models  to  predict  spatially  explicit  site  impacts.  After  a  decision  to  field  weapon  systems  is  made 
at  an  installation,  vehicle  tracking  systems  provide  both  a  new  capability  and  a  proactive  means 
for  the  installation  to  monitor  land  condition  and  preemptively  implement  LRAM  programs.  The 
direct  monitoring  of  mission  impacts  allows  land  managers  to  locate  the  most  severe  impacts 
after  training  events  and  mitigate  initial  site  damage  before  lands  further  degrade. 

2.2  OBJECTIVE  OF  THE  DEMONSTRATION 

The  objective  of  this  project  was  to  demonstrate  and  validate  the  VDMTS  system  and  its 
components  through  a  series  of  controlled  field  studies  and  live  tracking  events.  A  controlled 
field  study  was  used  to  demonstrate  and  validate  that  the  hardware  can  sufficiently  characterize 
vehicle  dynamic  properties  (turning  radius  and  velocity)  to  accurately  predict  site  impacts  (area 
impacted,  vegetation  loss,  and  RD).  A  controlled  field  study  was  used  to  demonstrate  and 
validate  the  accuracy  of  VDMTS  impact  models  in  predicting  area  impacted,  vegetation  loss,  and 
RD  for  a  range  of  vehicles.  Field  studies  tracking  live  training  exercises  and  subsequent  field 
measurements  were  used  to  demonstrate  and  validate  the  VDMTS  hardware  and  model 
perfonnance  in  predicting  site  impacts. 

2.3  REGULATORY  DRIVERS 

NEPA  requires  federal  agencies  to  evaluate  the  environmental  implications  of  their  plans, 
policies,  programs,  and  projects,  at  the  same  time  traditional  economic  and  technical  evaluations 
are  underway.  The  Federal  Water  Pollution  Control  Act,  commonly  referred  to  as  the  Clean 
Water  Act  (CWA),  is  intended  to  restore  and  maintain  the  chemical,  physical,  and  biological 
integrity  of  the  nation’s  waters  by  preventing  point  and  nonpoint  pollution  sources.  Off-road 
vehicle-based  maneuver  training  is  a  major  contributor  to  accelerated  erosion  on  military  lands 
and  can  be  in  violation  of  the  CWA.  Section  7  of  the  Endangered  Species  Act  (ESA)  requires 
federal  agencies  to  ensure  that  the  actions  they  take,  including  those  they  fund  or  authorize,  do 
not  jeopardize  the  existence  of  any  listed  species.  If  a  federal  agency  action  is  important  (i.e.,  live 
training),  the  motivation  is  for  the  federal  landowner  to  provide  data  to  support  their  position. 


3 


This  page  left  blank  intentionally. 


3.0  TECHNOLOGY/METHODOLOGY  DESCRIPTION 


This  section  describes  the  VDMTS  vehicle-based  maneuver  impact  assessment  process.  A 
description  of  the  technology  and  theoretical  models  utilized  in  the  system  is  given.  This  section 
also  summarizes  the  advantages  and  disadvantages  of  utilizing  the  VDTMS  process  as  opposed 
to  alternative  technologies. 

3.1  TECHNOLOGY/METHODOLOGY  OVERVIEW 

The  VDMTS  approach  is  used  to  assess  and  predict  impacts  resulting  from  military  vehicle- 
based  maneuver  training.  The  approach  consists  of  vehicle  impact  models,  vehicle  tracking 
hardware  and  software,  and  vehicle  tracking  data  analysis.  Each  component  of  the  VDMTS 
approach  is  discussed  in  more  detail  in  the  following  subsections. 

3.1.1  Vehicle  Tracking  Processes 

The  VDMTS  process  is  illustrated  in  Figure  1.  The  process  uses  vehicle  tracking  systems  to 
detennine  vehicle  location  and  dynamic  operating  characteristics  (i.e.,  turning  radius  and 
velocity).  The  VDMTS  vehicle  hardware  is  GPS-based  with  inertial  sensors  (Cybernet,  2002; 
2004).  This  component  of  the  VDMTS  system  is  shown  in  the  activity  characterization  step  of 
Figure  1.  Alternative  methods  such  as  custom-built  vehicle  tracking  systems  with  commercial 
off-the-shelf  components  and  military  standard  training  systems  (Army’s  BFT  and  National 
Guard’s  DFIRST)  could  also  be  used  to  collect  the  vehicle  dynamic  operating  characteristics 
(Anderson  et  al.,  2009;  Svendsen  et  al.,  2011). 


Figure  1.  Vehicle  tracking  and  impact  analysis  approach. 
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Process-based  vehicle  impact  models  were  developed  through  a  series  of  controlled  replicated 
studies  (Ayers  et  ah,  2005;  Ayers  et  ah,  2006;  Foster  et  ah,  2006;  Haugen  et  ah,  2003).  Impact 
models  were  developed  from  the  field  data  (Li  et  ah,  2007a;  Li  et  ah,  2007b).  The  process-based 
impact  models  are  indicated  in  the  Impact  Characterization  step  in  Figure  1.  An  alternative 
method  of  detennining  impacts  is  to  implement  the  system  used  prior  to  the  development  of  the 
theoretical  models:  perfonning  field  impact  assessments  and  developing  site-specific  regression 
models.  VDTMS  data  analysis  routines  are  used  to  post-process  tracking  data.  Analysis  routines 
include  spatial  displays  of  estimated  vegetation  loss  and  soil  rutting,  percent  of  vegetation  lost 
within  management  areas,  percent  on-  and  off-road  traffic,  potential  trail  identification 
(Anderson  et  ah,  2007a, b;  Ayers  et  ah,  2005;  Haugen  et  ah,  2003;  Rice  et  ah,  2006;  Wu  et  ah, 
2004;  Wu  et  ah,  2006;  Wu  et  ah,  2007). 


3.1.2  Vehicle  Impact  Models 

The  process-based  impact  models  predict  terrain  impacts  caused  by  wheeled  and  tracked  vehicles 
in  terms  of  percent  vegetation  cover  loss  (IS),  DW,  and  RD.  Percent  vegetation  loss  is  the 
primary  measure  of  site  impact  because  it  is  a  primary  variable  used  in  Army  operational 
monitoring  programs  and  an  input  variable  to  many  ecological  models.  DW  is  required  to 
convert  linear  distance  traveled  to  an  area  impacted.  RD  is  estimated  because  this  variable  is 
highly  correlated  with  vegetation  recovery  rates  and  is  important  to  models  that  incorporate 
microtopography.  Individual  models  were  developed  for  tracked  vehicles  and  for  four-,  six-,  and 
eight- wheeled  vehicles.  Equation  1  shows  the  DW  model  for  tracked  vehicles.  The  full 
derivation  of  the  tracked  model  can  be  found  in  Li  et  al.  (2007b).  A  similar  process-based, 
theoretical  model  was  developed  for  DW  from  wheel  vehicles.  DW  equations  and  their 
derivation  for  wheeled  vehicles  are  documented  in  Li  et  al.  (2007a). 


DW 


=  (T±  +  J^)+(tr-°  +  ™)-(tr-b--™) 

V  V  2  4 gUyTRj  V  2  2/  V  2  2  ) 


(1) 


Process-based  theoretical  models  were  developed  to  estimate  percent  vegetation  loss  based  on 
vehicle  type,  vehicle  dynamic  properties,  and  vehicle  static  properties.  Individual  models  were 
developed  for  tracked  vehicles  and  for  four-,  six-  and  eight-wheeled  vehicles.  Equation  2  shows 
the  vegetation  loss  model  for  tracked  vehicles.  The  full  derivation  of  the  tracked  model  can  be 
found  in  Li  et  al.  (2007b).  Higher  shear  stresses  produced  at  the  terrain  surface  result  in  more 
shear  displacement  and,  consequently,  greater  vegetation  loss.  The  vegetation  cover  and  the 
surface  soil  are  completely  scraped  away  when  the  shear  stress  reaches  the  maximum  strength 
that  the  soil  can  sustain. 

IS  —  (l  -  e-j/K~0-233)  x  100%  (2) 

Where: 

IS  =  vegetation  loss  (impact  severity),  [%] 
j  =  shear  displacement,  [m] 

K  =  the  shear  deformation  modulus,  [m] 
e  =  approximately  2.718 


Vegetation  loss  equations  and  their  derivation  for  wheeled  vehicles  are  documented  in  Li  et  al. 
(2007a).  A  critical  velocity,  vcn,  is  derived  by  balancing  soil  friction  forces  with  centrifugal 
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forces.  The  critical  velocity  is  defined  as  the  vehicle  speed  where  soil  shear  stress  is  equal  to  the 
soil  shear  strength.  The  vegetation  loss  is  calculated  from  a  ratio  of  actual  vehicle  velocity  to  the 
critical  velocity.  Any  increase  in  velocity  beyond  the  critical  velocity  results  in  the  vehicle 
sliding  laterally  and  complete  vegetation  removal. 

These  models  are  predictive  in  two  respects.  First,  vegetation  loss  is  not  measured  in  the  field  but 
predict  based  on  vehicle  characteristics  (static  and  dynamic)  and  site  conditions  (soil  strength). 
Second,  predictions  can  be  made  for  other  site  conditions  (i.e.,  wetter  or  dryer  conditions).  Using 
the  same  live  event  tracking  data,  predictions  can  be  made  for  vegetation  loss  in  wet  soils,  even 
if  the  event  may  have  occurred  during  dry  conditions.  This  prediction  assumes  that  wet 
conditions  do  not  change  the  pattern  of  vehicle  training,  only  the  magnitude  of  vegetation  loss. 

RD  or  sinkage  models  have  been  developed  by  the  U.S.  Army  Engineer  Research  and 
Development  Center  (ERDC).  The  Vehicle  Terrain  Interface  (VTI)  is  a  vehicle  terrain 
interaction  model  that  predicts  sinkage  for  vehicles  in  different  soil  conditions  (Richmond  et  ah, 
2004:  Jones  et  ah,  2007;  Nunez  et  ah,  2004).  However,  vehicle-operating  characteristics  (turning 
radius  [TR]  and  velocity)  are  not  inputs  in  this  model.  Liu  et  al.  (2010)  modified  the  VTI  model 
to  incorporate  weight  shift  due  to  changes  in  TR  and  velocity. 

3.1.3  Vehicle  Tracking  Hardware  and  Software 

A  low-cost  VDMTS  was  developed  to  automate  and  enhance  the  process  of  understanding  the 
spatial  and  temporal  characteristics  of  vehicle-based  impacts  for  assessing  land  condition, 
estimating  land  capacity,  and  restoring  lands  in  support  of  DoD  training  requirements  (Figure  2). 
The  VDMTS  hardware  consists  of  a  GPS  receiver  integrated  with  low-cost  micro  electro 
mechanical  systems  (MEMS)-based  strap  down  inertial  sensors.  These  sensors  enable 
measurement  of  vehicle  kinematics,  dynamics,  and  other  parameters  of  interest  that  enable 
accurate  modeling  of  environmental  impact.  The  system  thereby  provides  vehicle  dynamics  data 
and  positional  information  at  all  times,  even  when  GPS  is  unavailable.  The  VDMTS  has  the 
capability  to  record  the  vehicle  dynamics  tagged  with  position  infonnation  for  accurate  and 
enhanced  post-mission  analysis  at  substantially  reduced  costs.  Onboard  system  data  storage 
provides  archival  data  for  post-training  event  analysis. 


Figure  2.  VDMTS  hardware. 

The  photo  on  the  left  illustrates  the  hardware  in  the  durable  case.  The  photos  on  the  right 
illustrate  the  front  and  back  panels  of  the  VDMTS  unit. 
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3.1.4  Vehicle  Tracking  Data  Analysis  Routines 

Analysis  routines  were  developed  to  help  users  interpret  the  vehicle  tracking  data.  The 
assessment  of  individual  vehicle  impacts  and  impact  patterns  is  intended  to  address  management 
issues  like  the  ones  listed  below: 

1 .  How  much  damage  does  one  type  of  vehicle  cause  relative  to  other  vehicles? 

2.  How  much  damage  does  a  group  of  vehicles  conducting  a  training  exercise  cause 
relative  to  other  training  events? 

3.  Do  the  spatial  impact  patterns  for  individual  units  and  events  differ  from  the  historic 
patterns?  Should  we  expect  new  areas  to  be  impacted  that  have  not  historically  been 
impacted? 

4.  How  much  time  do  vehicles  spend  in  areas  designed  to  minimize  vehicle  impacts?  How 
much  time  is  spent  in  critical  habitat  or  near  critical  resources  like  cavity  trees? 

5.  Where  can  we  expect  new  trail  networks  to  form? 

Methods  were  developed  to  identify  areas  with  single  and  multiple  tracking  within  an  event  (Wu 
et  al.,  2004;  Rice  et  al.,  2006).  Tabular  summaries  identify  important  model  input  data  for  other 
environmental  models  (Anderson  et  al.,  2007a  and  b).  Another  example  is  percent  off-road  travel 
and  average  vegetation  cover  loss  per  mile  for  development  of  Anny  Training  and  Testing  Area 
Carrying  Capacity  (ATTACC)  Training  Impact  Factor  (TIF)  parameter  development  (Anderson 
et  al.,  2007a). 

3.2  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY/ 
METHODOLOGY 

The  VDMTS  approach  gives  installation  land  managers  a  tool  to  quantitatively  assess  impacts 
resulting  from  training  activities.  The  predictive  models  also  allow  assessment  of  training 
impacts  given  future  training  missions  and  weapon  systems.  Several  current  DoD  technologies 
exist  that  at  least  partially  meet  the  need  addressed  by  the  VDTMS.  The  VDTMS  approach  is 
limited  by  the  requirement  to  track  training  events  to  collect  required  data.  This  limitation  could 
be  relieved  by  utilizing  DoD  standard  training  and  tracking  systems  (e.g.,  DFIRST,  BFT)  to 
obtain  required  dynamic  vehicle  use  data. 

The  Army  Range  Requirements  Model  (ARRM)  is  a  planning  tool  that  models  training 
throughput  requirements  for  all  Army  installations.  It  allows  for  calculations  of  predicted 
training  miles  by  unit  and  event.  However,  the  training  data  are  only  at  the  installation  level  and 
do  not  allow  for  site  impact  predictions.  The  Range  Facility  Management  Support  System 
(RFMSS)  tool  is  a  range  and  training  area  scheduling  tool.  It  allows  for  quantification  of  training 
at  the  training  area  level;  however,  it  is  difficult  to  quantify  site  impacts  from  this  data.  The 
Range  and  Training  Land  Assessment  (RTLA)  program  monitors  training  land  condition  and 
land  use  intensity.  It  can  provide  temporally  and  spatially  explicit  patterns  of  training  impacts 
when  combined  with  ARRM  or  RFMSS  data.  Data  are  available  only  on  annual  time  intervals 
and  is  not  component  (training  event/unit)  defined  and  does  not  allow  for  estimation  of  future 
impacts  given  a  change  in  doctrine  or  weapon  system.  Installation  Specific  Vehicle  Impact 
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Studies  have  been  performed  for  NEPA  documentation.  These  studies  are  very  expensive  and  it 
is  often  difficult  to  extrapolate  data  to  actual  unit/event  impacts. 
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4.0  PERFORMANCE  OBJECTIVES 


Performance  objectives  are  provided  in  Table  1.  Performance  metrics  included  qualitative  and 
quantitative  parameters.  Quantitative  parameter  threshold  values  were  based  on  infonnation  from 
prior  studies.  For  example,  thresholds  for  accuracy  requirements  for  predicted  DW,  vegetation 
loss,  and  RD  were  based  on  1)  variation  typically  seen  in  field  measurements  of  the  variable; 
2)  variation  in  predicted  impacts  associated  with  variations  in  input  parameters;  3)  limitation 
introduced  from  other  sources  of  the  model,  hardware,  or  computations;  4)  attempt  to  balance 
data  collection  costs  and  data  quality;  and  5)  accuracy  required  to  make  management  decisions. 
The  metrics  are  based  on  data  that  were  collected  in  the  field.  Perfonnance  metrics  are  organized 
by  demonstration/validation  study  component  and  methodology  component. 
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Table  1.  Performance  objectives. 


Performance 

Objective 

Metric 

Data  Requirements 

Success  Criteria 

Results 

Quantitative  Performance  Objectives  -  Demonstration  Plan  for  Controlled  Field  Study  and  Live  Training  Single  Event  Study 

1.  Accurate  VDMTS 
hardware 
measurement  of 
vehicle  dynamic 
properties 

1.1.  VDMTS  hardware 
with  Inertial  Navigation 
System  (INS)  provides 
more  accurate  dynamic 
vehicle  properties  than 
GPS  alone. 

•  Vehicle  position  data 
without  GPS  signal 

•  Ability  to  record  in  situations 
when  GPS  signals  not  available 
due  to  topography,  vegetation, 
and  related  conditions 

•  Success  criteria  met:  Hardware  with  INS 
improved  dynamic  property 
measurement 

1.2.  VDMTS  hardware 
provides  sufficient 
dynamic  vehicle 
properties  to  predict 
vegetation  loss  and  soil 
rutting. 

•  Vehicle  positional 
accuracy 

•  Vehicle  turning  radius 
accuracy 

•  Vehicle  velocity 
accuracy 

•  Vehicle  positional  accuracy 
within  5  m  (16.4  ft)  95%  of  the 
time 

•  Vehicle  turning  radius  within  10 
m  (32.8  ft)  95%  of  the  time 

•  Vehicle  velocity  within  2.24  m 
per  second  (m/s)  (5  mph)  95%  of 
the  time 

•  Success  criteria  met:  Position  within  5  m 
99.9 %  of  recording  time 

•  Average  positional  accuracy  =  2.05  m 

•  Success  criteria  met:  TR  within  10  m 

95%  of  the  time 

•  Success  criteria  met:  Velocity  within 

2.24  m/s  100%  recording  time 

•  Average  velocity  error  =  -0.07  m/s  \ 

2.  Accurate  VDMTS 
impact  model 
predictions  of  site 
impacts  under 
controlled  condition 

2.1.  Correspondence 
between  predicted  and 
measured  DW, 
vegetation  loss,  and  RD 
of  site  damage 
associated  with 
individual  vehicle  use. 

•  DW 

•  Vegetation  loss 

•  RD 

•  Correlation  between  predicted 
and  measured  DW  >0.8 

•  Predicted  DW  within  20  cm  of 
actual  DW  for  95%  of  sample 
points 

•  Correlation  between  predicted 
and  measured  vegetation  loss 
>0.7 

•  Predicted  vegetation  loss  within 
20%  of  actual  vegetation  loss  for 
95%  of  sample  points 

•  Correlation  between  predicted 
and  measured  RD  >0.6 

•  Predicted  RDs  within  3  cm  of 
actual  RDs  for  95%  of  sample 
points 

•  Success  criteria  met:  Correlation 
between  predicted  and  measured  DW  = 
0.89 

•  Success  criteria  not  met:  Predicted  DW 
within  20  cm  of  actual  DW  for  50%  of 
sample  points 

•  Success  criteria  met:  Correlation 
between  predicted  and  measured 
vegetation  loss  >0.9 

•  Success  criteria  not  met:  Predicted 
vegetation  loss  within  20%  of  actual 
vegetation  loss  for  86%  of  sample 
points 

•  Success  criteria  not  met:  Correlation 
between  predicted  and  measured  RD 
<0.6 

•  Success  criteria  not  met:  Predicted  RDs 

within  3  cm  of  actual  RDs  for  94%  of 
sample  points  ' 

Table  1.  Performance  objectives  (continued). 


Performance 

Objective 

Metric 

Data  Requirements 

Success  Criteria 

Results 

3.  Accurate  VDMTS 
hardware 
measurement  of 
vehicle  static  and 
dynamic  properties 

3.1.  VDMTS  hardware 
provides  sufficient  static 
and  dynamic  vehicle 
properties  to  predict 
vegetation  loss  and  soil 
rutting  without  GPS 
signal. 

•  Vehicle  positional 
accuracy 

•  Vehicle  positional  accuracy 
within  10  m  (32.8  ft)  for  300  m 
(984.2  ft)  after  GPS  signal  lost 
90%  of  time 

•  Success  criteria  met:  Vehicle  positional 
accuracy  within  10  m  (32.8  ft)  for  300  m 
(984.2  ft)  after  GPS  signal  lost  100%  of 
time. 

•  Average  GPS  signal  error  1.6  m  (±0.1  m) 

•  Average  error  between  INS  and  GPS 

data  0.1 64±0. 002  m  , 

4.  Accurate  VDMTS 
model  predictions  of 
site  impacts  during 
live  training 

4.1.  Correspondence 
between  predicted  and 
measured  DW, 
vegetation  loss,  and  RD 
of  site  damage 
associated  with  vehicle 

use 

•  DW 

•  Vegetation  loss 

•  RD 

•  Predicted  DW  within  20  cm  of 
actual  D  W  in  90%  of  the  sample 
sites 

•  Predicted  vegetation  loss  within 
20%  of  actual  vegetation  loss  in 
80%  of  the  sample  sites 

•  Predicted  RD  within  4  cm  of 
actual  RD  in  80%  of  the  sample 
sites 

•  Success  criteria  not  met:  Predicted  DW 
within  20  cm  of  actual  DW  in  45.6%  of 
the  sample  sites 

•  Average  error  between  predicted  and 
measured  data  =14.9  cm 

•  Success  criteria  met:  Predicted 
vegetation  loss  within  20%  of  actual 
vegetation  loss  in  95%  of  the  sample 
sites 

•  Average  error  between  predicted  and 
measured  vegetation  loss  =  -1.8% 

•  Success  criteria  met:  Predicted  RD 
within  4  cm  of  actual  RD  in  100%  of  the 
sample  sites 

•  Average  error  between  predicted  and 

measured  RD  =  0.1  cm  , 

5.  VDMTS  hardware 
durability 

5.1.  Reliable  hardware 

use 

•  Percent  of  recording 
time  captured 

•  Percent  of  recording  time  >80% 
of  military  unit  training  time 

•  Success  criteria  met:  85.5%  of  military 
training  time  recorded.  , 

Table  1.  Performance  objectives  (continued). 
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Quantitative  Performance  Objectives  -  Demonstration  Plan  for  Live  Training  Multiple  Event  Study 

6.  VDMTS  hardware 
durability 

6.1.  Reliable  hardware 

use 

•  Percent  of  recording 
time  captured 

•  Percent  of  recording  time  >80% 
of  training  time  per  vehicle  type 
for  any  event 

•  Success  criteria  met:  90.2%  of  total 
training  time  recorded  (p  =  0.0002) 

•  >80%  training  time  by  vehicle  recorded 

except  for  heavy  expanded  mobility 
tactical  truck  (HEMTT)  , 

7.  Ease  of  system  use 

7.1.  Ability  of  a 
technician-level 
individual  to  install  and 
maintain  hardware 

•  Training  time 

•  Hardware  setup/take¬ 
down  time 

•  <4  hr/person 

•  <1  hr  total/vehicle  (setup  and 
take-down) 

•  Success  criteria  met:  0.3  hr/person 
(p<0.05) 

•  Success  criteria  met:  0.19  hr/vehicle 
(p<0.05) 

7.2.  Ability  of  a 
technician-level 
individual  to  retrieve 
and  quality 

assurance/quality  control 
(QA/QC)  data 

•  Training  time 

•  QA/QC  time 

•  <4  hr/person 

•  <1  hr/vehicle  data  file 

•  Success  criteria  met:  1.07  hr/person 
training  time  (p<0.05) 

•  Success  criteria  met:  0.82  hr/vehicle  data 
file  QA/QC  time  (p<0.05) 

7.3.  Ability  of  a 
technician-level 
individual  to  summarize 
results 

•  Training  time 

•  Analysis  time 

•  <16  hr/person 

•  <  8  hr/event  analysis 

•  Success  criteria  met:  6.33  hr/person 
training  time  (p<0.05) 

•  Success  criteria  met:  5.45  hr/event 
average  analysis  time  (p<0.05) 

8.  Quality  and 
accuracy  of  data  for 
land-use  decisions 

8.1.  Ability  to  use  data 
for  parameterization  of 
models 

•  Vehicle  position 

•  DW 

•  Vegetation  loss 

•  Time  offroad 

•  <10  m  position  error 

•  <20%  error  for  time  off-road,  and 
IS 

•  Success  criteria  met:  1.6  m  position 
accuracy 

•  Success  criteria  met:  6.3%  error  IS 

•  Success  criteria  met:  1.0%  error 

estimating  time  off-road  , 

8.2.  Ability  to  use  data 
to  identify  training  area 
use  patterns 

•  Vehicle  positional 
accuracy 

•  Vegetation  loss 

•  <10  m  position  error  for  LRAM 
identification  (ID) 

•  <5  m  position  error  for  threatened 
and  endangered  species  (TES) 
habitat  analysis 

•  <20%  error  in  vegetation  loss 

•  Success  criteria  met:  1.6  m  position  error 
for  LRAM  ID 

•  Success  criteria  met:  1.6  m  position  error 
for  TES  habitat  analysis 

•  Success  criteria  met:  6.3%  error 

estimating  vegetation  loss  : 

Table  1.  Performance  objectives  (continued). 
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9.  Ease  of  system  use 

9.1.  Ability  of  a 
technician-level 
individual  to  install, 
remove,  and  review 
collected  data 

•  Questionnaire  feedback 
from  the  technician  on 
usability  of  hardware 

•  Usable  hardware  system, 

QA/QC  process,  and  analyses 
processes 

•  Success  criteria  met:  Usable  hardware 
system,  QA/QC  process,  and  analyses 
processes 

10.  Quality  and 
accuracy  of  data  for 
land-use  decisions 

10.1.  Ability  to  use  data 
for  parameterization  and 
identifying  training  area 
use  patterns 

•  Questionnaire  feedback 
from  researchers  on 
usability  of  system 
results 

•  Usable  analysis  results 

•  Success  criteria  met:  Usable  analysis  | 

results 

This  page  left  blank  intentionally. 


5.0  SITE  DESCRIPTION 


This  section  provides  a  short  description  of  the  selected  demonstration  sites.  The  two  main 
demonstration  sites  are  Fort  Riley,  KS,  and  Fort  Benning,  GA,  with  additional  studies  performed 
at  Eglin  Air  Force  Base  (AFB),  FL,  and  Pohakuloa  Training  Area  (PTA),  HI.  The  combination 
of  the  four  sites  provides  a  broad  range  of  site  characteristics  (topography,  soil,  and  vegetation), 
vehicle  types,  and  training  doctrine. 

5.1  SITE  SELECTION 

For  this  study,  Fort  Riley  was  considered  the  alpha  test  site.  Fort  Riley  data  were  utilized  in 
original  impact  model  development  and  concept  development.  As  such,  use  of  impact  models  at 
this  location  provided  a  test  of  the  models  and  technology  within  the  model  development 
parameters.  Fort  Riley  also  represents  a  less  stressing  environment  in  that  it  is  relatively  flat  open 
grasslands  with  little  GPS  signal  interference.  Fort  Benning  was  considered  the  beta  test  site.  No 
data  from  Fort  Benning  were  used  in  the  development  of  the  model.  As  such,  this  demonstration 
site  represents  an  evaluation  of  the  models/technology  outside  the  original  bounds  of  model 
development.  Fort  Benning’s  topography  and  vegetation  are  more  diverse  allowing  for  a  more 
robust  evaluation  of  the  VDMTS’  GPS/INS  tracking  capabilities.  Eglin  AFB  was  chosen  as  an 
alternative  site  for  Fort  Benning  for  the  controlled  vehicle  impact  test.  Installation  land  managers 
at  PTA  expressed  interest  in  using  the  VDMTS  system  to  identify  vehicle  impacts  in  the  newly 
opened  Keamuku  Training  Area.  This  location  allowed  the  system  to  be  tested  and  validated 
under  different  conditions  and  on  different  vehicle  types. 

5.2  SITE  LOCATION,  HISTORY,  AND  SITE  CHARACTERISTICS 
5.2.1  Fort  Riley,  KS 

Fort  Riley  is  located  in  northeastern  Kansas,  USA  (Figure  3).  This  installation  has  an  area  of 
41,154  hectares  (ha).  It  is  located  in  the  Bluestem  Prairie  region  and  is  characterized  by  rolling 
plains  and  dissected  by  stream  valleys.  Installation  lands  are  a  mix  of  native  prairie  and 
introduced  vegetation.  Since  the  early  1940s,  a  variety  of  military  training  activities  including 
field  maneuvers,  combat  vehicle  operations,  mortar  and  artillery  fire,  and  small-arms  fire  have 
taken  place  at  Fort  Riley.  Typical  maneuvers  by  large  tracked  and  wheeled  vehicles  that  traverse 
thousands  of  hectares  in  a  single  training  exercise  can  cause  impacts  ranging  from  minor  soil 
compaction  and  lodging  of  standing  vegetation  to  severe  compaction  and  complete  loss  of 
vegetative  cover  in  areas  with  concentrated  training  use.  Figure  3  shows  the  study  areas  at  Fort 
Riley.  The  controlled  field  study  site  location  was  selected  to  have  access  to  installation 
representative  soil  and  vegetation  types  without  conflicting  with  ongoing  training  activities.  The 
live  training  event  study  site  location  identifies  the  region  most  commonly  used  by  maneuver 
training  events. 
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Figure  3.  Fort  Riley  controlled  field  study  and  live  training  event  study  locations. 

5.2.2  Fort  Benning,  GA 

Fort  Benning  is  located  in  southwest  Georgia  (Figure  4).  The  base  is  73,503  ha  in  size.  Most  of 
the  installation  lies  in  west  central  Georgia,  but  a  small  part  extends  into  Russell  County, 
Alabama.  The  installation  is  situated  on  the  Fall  Line  transition  zone,  which  is  the  geographic 
area  between  the  Southern  Appalachian  Piedmont  and  the  Coastal  Plain.  Soils  are  composed  of 
clay  beds,  weathered  Coastal  Plain  material,  and  alluvial  deposits  from  the  Piedmont  (Knowles 
and  Davo,  1997).  Open  areas  are  used  for  military  training  or  managed  for  wildlife  openings. 
The  military  open  areas  are  frequently  clear-cut  parcels  of  land  dominated  by  grass  and  bare  soil. 
Since  the  early  1920s,  land  impacts  due  to  light  and  heavy  military  activities  (e.g.,  infantry, 
artillery,  wheeled,  and  tracked  vehicle  training)  frequently  occur  in  open  areas.  Because  of  Base 
Realignment  and  Closure  (BRAC),  the  U.S.  Army  Annor  Center  and  School  is  currently  being 
relocated  to  Fort  Benning.  Heavy  military  training  will  be  increased  from  historical  levels, 
especially  in  the  southern  Good  Hope  Maneuver  Training  Area.  Figure  4  shows  the  study  areas 
at  Fort  Benning.  The  live  training  event  study  site  location  identifies  two  regions  most 
commonly  used  by  maneuver  training  events. 
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Figure  4.  Fort  Benning  controlled  field  study  and  live  training  study  locations. 

5.2.3  Eglin  Air  Force  Base,  FL 

Eglin  AFB  is  located  on  the  Florida  panhandle  (Figure  5).  It  is  the  largest  forested  military 
reservation  in  the  United  States  consisting  of  187,995  ha  within  Santa  Rosa,  Okaloosa,  and 
Walton  counties  (U.S.  Air  Force,  2010).  Eglin  AFB  has  a  subtropical  climate  characterized  by 
humid  warm  summers  and  mild  winters.  A  majority  of  the  soils  on  Eglin  AFB  belong  to  the 
Fakeland  Association  with  Fakeland  sand  the  dominate  soil  type.  Eglin’s  sandhills  are  comprised 
of  old  growth  longleaf  pine  forests  with  grasses,  forbs,  and  low  stature  shrub  groundcover.  This 
structure  is  maintained  by  frequent  fires  (3-5  year  frequency).  Eglin  AFB  is  the  largest  and  least 
fragmented  single  longleaf  pine  ownership  in  the  world. 


Figure  5.  Eglin  AFB  controlled  field  study  location. 


Controlled  Event  Study  Site 
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5.2.4  Pohakuloa  Training  Area,  HI 


PTA  is  located  on  the  island  of  Hawaii  (Figure  6).  It  is  53,735  ha  in  size  making  it  the  largest 
Anny  training  area  in  Hawaii  (U.S.  Anny  Environmental  Command,  2008).  The  installation  is 
located  in  the  saddle  between  Mauna  Loa  and  Mauna  Kea  volcanoes.  PTA  is  located  in  the 
Hawaiian  Islands  Province  of  the  Rainforest  Division  (Bailey,  1995).  Soils  on  PTA  are  thin  and 
poorly  developed.  The  predominant  soil  types  are  Keekee  loamy  sand  and  Kilohana  loamy  fine 
sand,  formed  in  volcanic  ash,  sand,  and  cinders.  Grassland,  shrubland,  and  treeland  make  up  the 
vegetation  communities  at  PTA.  PTA  is  utilized  for  maneuver  unit  live-fire,  maneuver  training, 
and  artillery  live-fire  (U.S.  Department  of  the  Army,  2002).  In  2006,  the  9210  ha  Keamuku 
Training  Area  was  added  to  PTA  to  support  battalion  maneuver  training  and  to  support  training 
of  the  Stryker  Brigade  Combat  Teams.  Figure  6  illustrates  the  study  areas  at  PTA.  The 
Controlled  Event  study  site  was  located  in  the  recently  acquired  Keamuku  Training  Area. 
Training  monitored  under  the  Live  Event  study  occurred  all  throughout  PTA,  including  the 
Keamuku  Training  Area. 


Controlled  Event  Study  Site 


■"  ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■. 

■  Live  Event  Study  Site  i 
. . . . . 


Keamuku  Training  Area 


Grassland/Herbaceous 
""|  Pasture  Hay 


|  Developed,  Open  Space 
|  Developed,  Low  Intensity 
|  Developed,  Medium  Intensity 
|  Developed,  Hign  Intensity 
|  Barren  Land 
|  Evergreen  Forest 
I  Shrub/Scrub 


Figure  6.  PTA  controlled  field  study  and  live  training  event  study  locations. 
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6.0 


TEST  DESIGN 


6.1  CONCEPTUAL  TEST  DESIGN 

The  conceptual  demonstration  test  design  is  illustrated  in  Figure  7.  The  demonstration  plan 
consisted  of  first  verifying  that  the  hardware  was  properly  recording  vehicle  dynamic  and  static 
properties  for  use  in  subsequent  tests.  The  second  test  consisted  of  demons trating/validating  the 
impact  models.  Field  measurements  of  site  impacts  were  used  to  assess  impact  model 
perfonnance.  The  third  test  consisted  of  installing  tracking  units  on  multiple  vehicles  during  a 
single  live  training  event.  Upon  completion  of  the  training  event,  vehicle  dynamic  properties  and 
locations  were  used  to  predict  impacts  along  vehicle  paths.  The  fourth  test  consisted  of  installing 
tracking  units  on  multiple  vehicles  during  multiple  live  training  events.  Data  were  summarized  to 
address  one  or  more  installation  identified  land  management  issues.  This  test  detennined  the  ease 
of  system  use  and  determined  the  applicability  of  the  VDMTS  unit  to  land  management  decision 
making. 


Figure  7.  Conceptual  demonstration  plan. 


6.2  BASELINE  CHARACTERIZATION  AND  PREPARATION 

Baseline  characterization  and  preparation  was  essentially  the  first  test  in  the  conceptual 
demonstration  plan  (Figure  7).  Baseline  characterization  consisted  of  verifying  that  the  hardware 
functioned  properly  and  recorded  vehicle  dynamic  and  static  properties  for  use  in  subsequent 
tests.  Tests  were  conducted  for  location,  velocity,  turning  radius  accuracy,  and  GPS  signal  loss 
(forcing  INS  system  perfonnance).  These  baseline  tests  were  performed  at  the  University  of 
Tennessee  and  the  methods  used  are  described  in  the  following  section. 
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6.3  DESIGN  AND  LAYOUT  OF  TECHNOLOGY  AND  METHODOLOGY 
COMPONENTS 

6.3.1  VDMTS  Hardware  Positional  Accuracy  Test 

VDMTS  systems  were  located  on  a  known  benchmark  (assumed  location  truth)  and  allowed  to 
collect  a  minimum  of  GPS  position  data,  Universal  Time  Coordinated  (UTC),  and  Horizontal 
Dilution  of  Precision  (HDOP)  for  at  least  6  hours.  The  data  were  transferred  from  the  log  files  to 
spreadsheet  files  for  data  analysis.  The  position  data  were  converted  to  the  Universal  Transverse 
Mercator  (UTM)  coordinate  system  for  analysis.  The  average  position  for  the  GPS  points  is 
detennined  by  finding  the  average  of  the  Northing  and  Easting  coordinates.  The  position  error 
(in  meters)  for  each  point  was  calculated.  The  average  of  the  position  errors  was  determined  and 
recorded. 

6.3.2  VDMTS  Hardware  Velocity  Accuracy  Test 

The  VDMTS  tracking  units  were  mounted  on  a  vehicle  that  was  driven  at  three  different  but 
constant  velocities  along  a  track  of  known  GPS  coordinates.  The  velocity  of  the  vehicle  was 
detennined  by  timing  the  vehicle  with  a  stopwatch  over  a  predetennined  distance.  The  timed 
velocity  was  assumed  to  be  truth.  The  velocity  was  also  calculated  from  GPS/INS  data  by 
detennining  the  change  in  position  data  of  the  vehicle.  The  same  three  controls  used  in  the 
location  test  were  also  used  in  the  velocity  test  for  comparison  purposes. 

6.3.3  VDMTS  Hardware  Turning  Radius  Accuracy  Test 

The  VDMTS  tracking  units  were  mounted  on  a  vehicle  that  was  driven  around  several  constant 
radius  courses  (differing  radii)  and  along  a  straight  path  multiple  velocities.  The  distance  from 
the  center  pivot  to  each  of  the  course  paths  was  used  as  the  actual  radius  (truth)  and  was 
compared  to  the  radius  values  calculated  from  the  position  data  provided  by  the  units.  The  same 
three  controls  used  in  the  location  test  were  also  used  in  the  velocity  test  for  comparison 
purposes. 

6.3.4  VDMTS  Hardware  INS  System  Test 

The  INS  subsystem  of  the  VDMTS  tracking  system  is  designed  to  ensure  vehicle  location  and 
dynamic  property  values  during  periodic  GPS  signal  loss  or  loss  of  GPS  signal  quality.  The  same 
three  controls  used  in  the  location  test  were  used  in  the  velocity  test  for  the  same  comparison 
purposes.  Tracking  units  were  mounted  on  a  vehicle  that  is  driven  through  tunnels,  vegetation, 
and  other  areas  of  poor  or  no  GPS  signal.  The  location,  velocity,  and  turning  radius  along  the 
course  with  and  without  signal  were  recorded. 

6.4  FIELD  TESTING 

6.4.1  Controlled  Impact  Model  Validation  Study 

A  series  of  field  studies  was  conducted  using  multiple  vehicles  (M1A1  Abrams  Tank,  Annored 
Personnel  Carrier  [APCJ-M113,  M2A2  Bradley,  High  Mobility  Multipurpose  Wheeled  Vehicle 
[HMMWV],  HEMTT,  and  Stryker  Light  Armored  Vehicle  [LAV]  III).  These  vehicles  covered  a 
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range  of  tracked  (light  to  heavy)  and  wheeled  vehicles  (light  to  heavy  and  multiple  axles).  Since 
the  impact  models  were  designed  for  use  with  various  types  of  vehicles  and  incorporate  vehicle 
static  properties  (weight,  track/wheel),  vehicles  that  represent  a  range  in  vehicle  static  properties 
were  appropriate  for  model  validation  testing.  Each  vehicle  was  driven  on  a  systematically 
planned  course  (spiral)  within  a  randomly  located  treatment  plot  (Figure  8).  Each  spiral  course 
within  a  treatment  plot  consisted  of  a  section  of  straight-line  travel  followed  by  a  section  of 
constantly  decreasing  turning  radius.  One  spiral  for  each  vehicle  was  traversed  at  a  slow  and  fast 
velocity.  A  VDMTS  hardware  unit  was  mounted  on  each  vehicle  tested. 


Figure  8.  Controlled  field  test  study  design. 

Spirals  show  vehicle  courses.  Dots  show  measurement  plots. 
Arrow  shows  an  example  of  a  sample  point. 

The  line  across  the  vehicle  track  illustrates  a  measurement  transect. 


6.4.2  Live  Training  Test  -  Model  Validation 

A  held  study  was  conducted  by  tracking  a  live  training  event  using  VDMTS  systems  at  Fort 
Riley  and  Fort  Benning.  Military  training  events  were  identified  through  coordination  with 
installation  personnel.  The  military  training  events  tracked  were  based  on  availability  of  units 
training  at  the  installation,  access  to  unit  equipment,  duration  of  training,  number  of  vehicles, 
types  of  vehicles,  mission  doctrine,  training  areas  scheduled,  and  installation  objectives  for 
study.  The  events  included  1)  four  vehicle  types,  2)  on  and  off  road  activity,  3)  wide  range  of  site 
conditions  utilized,  and  4)  5-  to  10-day  durations. 

A  number  of  vehicles  were  instrumented  (Table  2)  with  VDMTS  hardware  at  both  sites. 
Tracking  units  were  installed  in  motor  pool  and  removed  after  the  event  was  completed.  VDMTS 
data  were  used  to  locate  vehicle  tracks  within  a  few  days  of  the  completion  of  training.  Sample 
locations  were  randomly  located  along  vehicle  paths.  Measurements  of  site  damage  (DW,  IS, 
RD)  were  made  at  each  sample  point  using  methods  described  in  Section  6.5,  below.  Predicted 
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site  damage  (DW,  IS,  RD)  were  compared  with  measured  site  damage  to  quantify  the  ability  of 
VDMTS  to  predict  overall  site  damage  for  a  training  event. 

Table  2.  Vehicles  tested  during  live  training  event  model  validation. 


Site 

Test  Dates 

Vehicle  Types  Tested 

# 

Vehicles 

Tested 

Fort  Riley 

17-21  Aug  2009 

HMMWV,  Buffalo,  Medium  Tactical  Vehicle  (MTV) 

18 

Fort  Benning 

31  Oct -9  Nov  2011 

HMMWV,  Stryker  LAV  III 

20 

6.4.3  Live  Training  Test  -  Multiple  Events  for  System  Validation 

A  field  study  was  conducted  by  tracking  a  series  of  live  training  events  using  VDMTS  systems 
(Table  3).  A  series  of  military  training  events  was  identified  through  coordination  with 
installation  personnel.  The  military  training  events  tracked  were  based  on  an  installation  defined 
issue  that  relates  to  vehicle  impacts  on  installation  lands.  A  minimum  of  four  training  events 
were  tracked  per  installation  with  a  total  of  14  events  tracked  over  3  years.  For  each  training 
event,  approximately  10-20  vehicles  were  instrumented  and  tracked  depending  on  the  training 
event  selected  and  the  number  of  vehicles  in  each  event. 


Table  3.  Events  tracked  during  multiple  live  training  events  phase  of  study. 


Site 

Test  Dates 

#  Days 

Vehicle  Types 

#  Vehicles 

Fort  Riley 

17-21  Aug  2009 

5 

HMMWV,  Buffalo,  MTV 

18 

Fort  Riley 

13-15  Jul  2010 

3 

HMMWV,  MTV 

7 

Fort  Riley 

10-17  May  2011 

8 

HMMWV,  HEMTT,  Light  Medium 
Tactical  Vehicle  (LMTV) 

12 

Fort  Riley 

17-22  May  2011 

6 

HMMWV,  HEMTT,  LMTV 

11 

Fort  Benning 

18-20  Oct  2010 

3 

Stryker,  Bradley 

9 

Fort  Benning 

28-29  Mar  20 11 

2 

Stryker,  Bradley,  M1A1,  HMMWV 

7 

Fort  Benning 

31  Oct -9  Nov  2011 

10 

Stryker,  HMMWV 

20 

Fort  Benning 

9-14  Nov  2011 

6 

Stryker,  HMMWV 

22 

PTA 

6-9  Nov  2009 

4 

Stryker 

3 

PTA 

24-29  Jan  2010 

6 

Amphibious  Assault  Vehicle  (AAV) 

6 

PTA 

17-23  Jan  2011 

6 

AAV 

6 

PTA 

8-10  Jun  2011 

3 

AAV 

2 

PTA 

13-14  Jun  2011 

3 

AAV 

6 

PTA 

16-17  Jun  2011 

2 

AAV 

7 

6.5  SAMPLING  PROTOCOL 

6.5.1  Sampling  Protocol  for  Metric  Evaluation 

6.5.1. 1  Controlled  Event  Study 

Vehicle  impacts  (i.e.,  DW,  IS,  RD)  were  measured  immediately  after  tracking.  DW  was 
measured  perpendicular  to  the  vehicle  track  and  encompassed  the  area  where  soil  or  vegetation 
was  impacted  by  the  vehicle  tire  track.  IS  was  estimated  using  a  line  transect  established 
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perpendicular  to  the  track.  A  second  line  transect  was  established  perpendicular  to  the  track  and 
adjacent  to  the  track.  For  each  line  transect  (within  track  and  adjacent  to  the  track),  bare  ground 
was  visually  estimated  and  reported  as  a  percent  of  transect  length.  IS  is  the  difference  between 
the  two  values.  RD  was  measured  using  a  ruler  laid  horizontally  across  the  outside  track  from 
undisturbed  soil  on  the  inside  of  the  track  to  undisturbed  soil  on  the  outside  of  the  track.  A 
second  ruler  measured  the  deepest  portion  of  the  rut. 

6.5.1.2  Model  Validation  Live  Training  Event  Study 

Primary  field  data  collection  consisted  of  using  high  quality  GPS  units  to  follow  previously 
driven  vehicle  paths.  Sample  locations  were  based  on  the  GPS  location  and  visual  location  of  the 
off-road  vehicle  track.  This  sampling  allowed  for  an  unbiased  comparison  of  actual  and  predicted 
vegetation  loss,  IS,  and  rutting.  DW,  IS,  and  RD  were  measured  as  described  above.  Sample 
points  were  measured  as  paired  sample  locations  within  the  track  and  outside  of  the  track. 

6.5.1.3  Multiple  Live  Training  Event  Study 

A  subset  of  the  total  number  of  events  scheduled  for  the  installation  was  sampled  at  each 
location.  A  subset  of  the  total  number  of  vehicles  in  an  event  was  evaluated  so  the  subset  of 
vehicles  was  representative  of  the  whole  training  event.  This  allowed  data  from  the  subset  to  be 
extrapolated  to  the  whole  event.  The  sampling  accounted  for  differences  in  units  and  vehicle 
types.  Military  personnel  provided  input  on  vehicle  use  as  related  to  doctrine.  Natural  resources 
personnel  helped  related  vehicle  selection  to  decision-making  process. 

6.6  SAMPLING  RESULTS 

6.6.1  Hardware  Tests 

The  first  component  of  the  controlled  field  study  demonstration  plan  was  to  validate  that  the 
hardware  was  properly  functioning  and  recording  vehicle  dynamic  and  static  properties  for  use  in 
subsequent  tests.  The  VDMTS  system  initially  was  the  worst  performing  of  the  five  systems 
tested  (Table  4).  This  was  because  the  GPS  signal  in  the  initial  VDMTS  system  was  not 
differentially  corrected.  The  other  units  tested  all  used  differential  correction  to  increase  the 
positional  accuracy.  A  GPS  system  capable  of  Wide  Area  Augmentation  System  (WAAS) 
differential  correction  replaced  the  previous  GPS  system  in  the  VDMTS.  The  static  position 
evaluation  test  was  repeated.  Table  5  gives  a  summary  of  the  test  results  using  the  upgraded 
VDMTS  system. 


Table  4.  Initial  position  evaluation  tests. 


Metric 

VDMTS 

(m) 

INS/GPS 

(m) 

G18 

(m) 

T132 

(m) 

T332 

(m) 

Average 

7.39 

4.25 

2.48 

1.72 

0.06  | 

CEP 

6.94 

2.58 

2.58 

1.14 

0.06  ! 

2DRMS 

12.13 

3.90 

3.90 

4.55 

0.10  ; 

CEP  =  circular  error  probable 

2DRMS  =  twice  distance  root  mean  square 
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Table  5.  Upgraded  VDMTS  position  evaluation  tests. 


Metric 

VDMTS 

(upgraded) 

(m) 

VDMTS 

(initial) 

(m) 

G18  (#1) 

(m) 

G18  (#2) 

(m) 

T132 

(m) 

Avg.  error 

2.05 

7.39 

2.11 

2.75 

0.28 

CEP 

2.00 

6.94 

1.89 

3.04 

0.20 

2DRMS 

3.71 

12.13 

3.74 

4.44 

0.64 

Avg.  HDOP 

0.9 

NA 

1.1 

1.0 

1.1 

Avg.  SAT  No. 

10.5 

NA 

8.3 

8.3 

7.7 

DGPS  % 

100 

NA 

100 

100 

100 

SAT  =  satellite 

DGPS  =  Differential  Global  Positioning  System 


A  velocity  evaluation  test  was  performed  on  the  VDMTS  system  as  described  in  Section  6.3.2. 
The  results  from  the  velocity  evaluation  test  are  presented  in  Table  6.  The  truth  value  represents 
the  actual  measured  velocity  while  the  VDMTS  values  represent  the  VDMTS  calculated. 

Table  6.  Velocity  evaluations. 


Test 

Truth 

(m/s) 

VDMTS 

(m/s) 

Error 

(m/s) 

Error  % 

1 

1.43 

1.47 

-0.04 

3.00 

2 

1.62 

1.76 

-0.14 

8.64 

3 

1.40 

1.43 

-0.03 

2.19 

Average 

1.48 

1.55 

-0.07 

4.80 

Accuracy  of  turning  radius  was  measured  by  mounting  the  units  on  a  cart  pushed  at  two 
velocities  on  the  track  with  a  range  of  known  turning  radii  as  described  in  Section  6.3.3.  The  data 
collected  are  illustrated  in  Figure  9  with  the  results  summarized  in  Table  7. 
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Figure  9.  Turning  radius  evaluation  tests. 
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Table  7.  Turning  radius  evaluation  tests. 


Actual  TR 

VDMTS 

INS/GPS 

G18 

T132 

T332 

Velocity 

(m) 

(m) 

(m) 

(m) 

(m) 

(m) 

Slow 

6.7 

6.5 

4.0 

4.0 

Slow 

10.5 

10.8 

6.7 

6.1 

Slow 

18.6 

20.0 

10.9 

14.1 

Slow 

38.0 

62.3 

54.0 

53.8 

62.0 

49.7 

Slow 

48.0 

48.4 

59.0 

47.2 

65.8 

72.3 

Fast 

38.0 

49.4 

39.8 

48.8 

38.9 

39.3 

Fast 

48.0 

58.5 

46.1 

55.3 

50.2 

54.4 

The  final  success  criteria  for  the  first  perfonnance  objective  was  to  detennine  ability  to  record  in 
situations  when  GPS  signals  were  not  available  due  to  topography,  vegetation,  and  related 
conditions.  This  was  tested  by  driving  the  unit  through  a  tunnel  and  under  canopy.  Results  show 
that  the  INS  data  collected  from  the  Vehicle  Dynamics  Monitor  (VDM)  hardware  increase  the 
accuracy  when  compared  to  systems  without  INS  capability  (Table  8). 

Table  8.  Positional  error  under  heavy  cover  test. 


GPS  System 

Average  Distance  Error 
(m) 

Standard  Deviation 
(m) 

Trimble  (without  INS) 

2.7 

0.25 

Garmin  (without  INS) 

2.7 

0.34 

VDM 

2.2 

0.17 

6.6.2  Controlled  Test  Model  Validation 

The  main  objective  of  the  controlled  test  was  to  validate  that  the  impact  models  accurately 
predicted  site  impacts  based  on  vehicle  dynamic  properties.  Tests  conducted  included  DW, 
vegetation  loss,  and  RD  measurements.  These  tests  were  performed  using  multiple  vehicles  at 
Fort  Riley,  PTA,  and  Eglin  AFB.  The  theoretical  model  results  were  compared  with  measured 
impact  values.  Additionally,  measured  impact  values  were  used  to  develop  site,  vehicle,  and 
condition  specific  statistical  models  (Figure  10).  These  statistical  models  represent  the  best 
prediction  possible  with  the  variability  observed  and  are  a  current  method  of  impact  assessment. 
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Figure  10.  Example  of  statistical  regression  model  developed  for  Stryker  vehicle  at  PTA. 

A  linear  regression  of  the  predicted  versus  measured  data  was  performed  to  determine  the 
closeness  to  a  unity  slope.  Using  this  as  an  indicator  of  model  quality,  the  DW  statistical  model 
perfonned  slightly  better  than  the  theoretical  model  at  Eglin  AFB  and  PTA  but  not  at  Fort  Riley 
(Figures  11-13).  Using  linear  regression  as  an  indicator  of  model  success,  the  statistical  IS 
(vegetation  removal)  models  slightly  outperformed  the  theoretical  models  for  the  controlled 
studies  (Figures  14-16).  Given  the  intended  use  of  the  models,  a  more  appropriate  measure  of 
model  validation  is  the  average  error  between  the  predicted  and  measured  impacts  using  each 
model  (Table  9). 


Measured  DW  (cm) 


Figure  11.  Theoretical  and  statistical  model  predicted  DW  values  compared  with  measured 

values  for  Fort  Riley  controlled  study. 
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Figure  12.  Theoretical  and  statistical  model  predicted  DW  values  compared  with  measured 

values  for  Eglin  AFB  controlled  study. 
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Figure  13.  Theoretical  and  statistical  model  predicted  DW  values  compared  with  measured 

values  for  PTA  controlled  study. 
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Measured  IS  (%) 

Figure  14.  Theoretical  and  statistical  model  predicted  IS  values  compared  with  measured 

values  for  Fort  Riley  controlled  study. 
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Figure  15.  Theoretical  and  statistical  model  predicted  IS  values  compared  with  measured 

values  for  Eglin  AFB  controlled  study. 
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Figure  16.  Theoretical  and  statistical  model  predicted  IS  values  compared  with  measured 

values  for  PTA  controlled  study. 

Table  9.  Average  absolute  error  between  predicted  and  measured  values 
for  theoretical  and  statistical  models. 

(Note:  Units  are  cm  for  DW,  RD  and  percentage  for  IS) 


Riley 

Eglin  AF 

B 

PTA 

DW 

IS 

RD 

DW 

IS 

RD 

DW 

IS 

RD 

Theoretical 

Average  absolute  error 

22.0 

9.8 

1.1 

33.9 

9.8 

0.1 

22.4 

13.4 

0.7  | 

Average  error 

-7.7 

-3.7 

1.0 

17.4 

-8.2 

-0.1 

-16.2 

11.9 

0.6 

Statistical 

Average  absolute  error 

13.0 

11.5 

3.9 

24.2 

7.5 

0.9 

14.8 

10.8 

1.0 

Average  error 

1.5 

1.9 

-3.9 

-0.8 

0.5 

-0.9 

2.3 

3.3 

0.1 

The  statistical  models  predicted  DW  and  IS  better  than  the  theoretical  models  in  five  of  the  six 
instances.  This  observation  is  expected  since  the  data  were  actually  used  to  develop  the  statistical 
models  in  this  case.  A  separate  dataset  would  need  to  be  collected  to  assess  unbiased  statistical 
model  performance  under  the  controlled  study. 

6.6.3  Single  Live  Training  Event  Study  and  Model  Validation 

VDTMS  hardware  and  models  were  tested  in  live  training  events  by  tracking  a  live  training 
event  at  Fort  Riley  and  Fort  Benning.  Measured  impacts  from  these  tracks  were  compared  with 
model  predicted  values.  The  statistical  models  developed  at  both  sites  were  also  used  to  predict 
impacts  (Figures  17-20).  Even  though  the  regression  between  predicted  and  measured  was  not  as 
good  for  the  live  event  study,  the  average  absolute  errors  for  the  theoretical  models  in  the  live 
training  event  study  are  comparable  to  the  errors  measured  in  the  controlled  study  (Table  10). 
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Figure  17.  Theoretical  and  statistical  model  predicted  DW  values  compared  with  measured 

values  for  Fort  Riley  live  training  event  study. 
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Figure  18.  Theoretical  and  statistical  model  predicted  DW  values  compared  with  measured 
values  for  Fort  Benning  live  training  event  study. 
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Figure  19.  Theoretical  and  statistical  model  predicted  IS  values  (vegetation  removal) 
compared  with  measured  values  for  Fort  Riley  live  training  event  study. 


Measured  IS  (%) 

Figure  20.  Theoretical  and  statistical  model  predicted  IS  values  (vegetation  removal) 
compared  with  measured  values  for  Fort  Benning  live  training  event  study. 
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Table  10.  Absolute  average  errors  between  predicted  and  measured  impacts 

for  live  training  events. 


Fort  Riley 

Fort  Benning 

Combinec 

DW 

IS 

RD 

DW 

IS 

RD 

DW 

IS 

RD 

Theoretical 

24.8 

7.7 

0.7 

34.8 

4.9 

0.0 

29.7 

6.3 

Statistical 

27.2 

12.9 

0.9 

32.6 

6.4 

0.5 

29.7 

8.3 

While  the  statistical  models  performed  better  than  the  theoretical  models  in  the  controlled  study, 
the  theoretical  models  perfonned  as  well  as  statistical  models  in  the  live  single  training  event 
study.  In  practice,  this  means  that  the  theoretical  models  may  perfonn  as  well  as  site-specific 
statistical  models  in  estimating  impacts  from  a  military  training  event. 


In  addition  to  modeling  performance,  hardware  durability  and  positional  accuracy  were  assessed 
in  the  single  live  training  event  tests.  Table  1 1  summarizes  the  total  training  time,  recorded  time, 
and  percent  of  total  training  time  for  each  training  event. 

Table  11.  Summary  of  hardware  durability  (training  time  recorded) 
from  single  live  training  events. 


Installation 

Total  Time 
(hr) 

Recorded  Time 
(hr) 

%  Recorded 

Fort  Riley 

1800.0 

1489.1 

82.7 

Fort  Benning 

209.6 

209.7 

100.0 

PTA 

628.0 

556.0 

88.5 

TOTAL 

2637.6 

2254.8 

85.5 

The  positional  accuracy  was  calculated  by  comparing  the  location  of  the  tracks  in  the  field  with 
the  VDMTS  collected  data  at  Fort  Riley  and  Fort  Benning.  At  Fort  Riley,  the  positional  accuracy 
observed  was  1.6  m  (±0.1  m).  The  positional  accuracy  observed  at  Fort  Benning  was  1.1m  (±0.1 
m).  To  quantitatively  assess  INS  performance  without  GPS  signal,  data  were  analyzed  with  and 
without  INS  data  being  forced  to  the  GPS  signal.  This  allowed  a  measurement  of  INS  data  error 
compared  with  the  GPS  signal.  A  subset  of  10  out  of  the  38  single  live  training  event  vehicle 
files  (48,259  sampling  points)  was  analyzed  for  INS  perfonnance.  In  the  subset  of  10  vehicles, 
the  positional  accuracy  never  exceeded  the  10  m  threshold  signifying  the  success  criterion  was 
met.  The  average  error  compared  with  the  GPS  signal  was  0.164±0.002  m.  Across  the  48,259 
samples,  the  maximum  error  compared  with  the  GPS  signal  was  2.60  m. 

6.6.4  Multiple  Live  Training  Event  Tests 

6.6.4.1  Multiple  Live  Training  Event  Hardware  Durability 

Hardware  durability  was  assessed  through  measuring  the  percent  of  each  training  event  that  was 
recorded  by  the  vehicular  tracking  systems.  Any  loss  of  data  was  attributed  to  hardware  error 
(e.g.,  hardware  breakage,  power  failure,  data  card  fault,  loss  of  hardware).  Table  12  summarizes 
the  total  time  recorded  and  total  training  time  recorded  for  each  event.  A  total  of  13,587.1  hr  of 
data  was  recorded  out  of  15,056.8  hr  of  total  training  time  during  which  the  units  were  mounted 
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(90.2±2.5%).  In  addition,  recorded  time  was  analyzed  by  vehicle  type  to  determine  if  hardware 
durability  is  a  function  of  vehicle  type  (Table  13).  Except  for  the  HEMTT,  over  80%  of  the  data 
were  recorded  for  each  vehicle. 


Table  12.  Hardware  durability  performance  by  event  (%  of  training  time  recorded). 


Installation 

Date 

#  Vehicles 

Total  Time 
(hr) 

Recorded  Time 
(hr) 

%  Recorded 

Fort  Riley 

17-21  Aug  2009 

18 

1800.0 

1489.1 

82.7 

Fort  Riley 

13-15  Jul  2010 

7 

332.6 

250.9 

77.8 

Fort  Riley 

10-17  May  2011 

12 

2108.2 

1682.6 

79.8 

Fort  Riley 

17-22  May  2011 

11 

1515.9 

1225.6 

80.8 

Fort  Benning 

18-20  Oct  2010 

9 

519.5 

519.2 

99.9 

Fort  Benning 

28-29  Mar  20 11 

7 

209.6 

209.7 

100.0 

Fort  Benning 

31  Oct -9  Nov  2011 

20 

4215.2 

3998.6 

94.9 

Fort  Benning 

9-14  Nov  2011 

22 

2700.3 

2700.3 

100 

PTA 

6-9  Nov  2009 

3 

86.9 

86.9 

100 

PTA 

24-29  Jan  2010 

6 

509.9 

471.8 

92.5 

PTA 

17-23  Jan  2011 

6 

628.0 

556.0 

88.5 

PTA 

8-10  Jun  2011 

2 

49.2 

49.2 

100.0 

PTA 

13-14  Jun  2011 

6 

171.3 

143.6 

83.8 

PTA 

16-17  Jun  2011 

7 

220.2 

2 

92.5 

TOTAL 

139 

15056.8 

13587.1 

90.2±2.3 

Table  13.  Hardware  durability  performance  by  vehicle  type  (%  of  training  time  recorded). 


Vehicle 

Type 

Number 

Vehicles 

Total  Time 
(hr) 

Recorded 
Time  (hr) 

%  Recorded 

HMMWV 

Wheeled 

59 

8085.6 

7597.9 

93.9±2.9 

MTV 

Wheeled 

7 

649.4 

550.5 

84.8±1 1.3 

Buffalo 

Wheeled 

1 

71.9 

71.9 

100.0±0.0 

Stryker 

Wheeled 

18 

1785.2 

1785.2 

99.98±0.0 

Bradley 

Tracked 

6 

289.7 

289.7 

100.0±0.0 

AAV 

Tracked 

26 

1578.6 

1424.2 

90.2±4.6 

Abrams 

Tracked 

1 

30.1 

30.1 

100±0.0 

HEMTT 

Wheeled 

12 

1881.4 

1181.2 

62.79±13.9 

LMTV 

Wheeled 

4 

626.4 

626.4 

100.0±0.0 

6.6.4.2  Multiple  Live  Training  Event  Ease  of  Use 

A  second  objective  of  the  multiple  live  training  event  demonstration  was  to  assess  the  time 
requirements  for  implementing  the  VDMTS  process.  As  described  in  Section  6.4.3,  the  time 
required  for  training  technicians  on  each  step  of  the  VDMTS  as  well  as  time  required  to  perfonn 
each  step  were  recorded.  Table  14  documents  the  number  of  technicians  trained  for  each  step  and 
the  training  time  required  for  each  step.  Table  15  exhibits  the  time  required  to  perfonn  each  step 
of  the  VDMTS  process  and  summarizes  the  number  of  vehicles  and  events  for  which  these  data 
were  collected. 


35 


Table  14.  Training  times  for  each  step  of  VDMTS  implementation 
for  ease-of-use  assessment. 


Ease-of-Use  -  Training  Times 

Hardware  Use 

Data  QA/QC 

Analysis 

#  Technicians  trained 

16 

6 

6 

Average  time  (hr) 

0.3 

1.07 

6.33 

Standard  deviation  (hr) 

0.12 

0.11 

0.82 

Standard  error  (hr) 

0.23 

0.35 

2.58 

Table  15.  Performance  times  for  each  step  of  VDMTS  implementation 

for  ease-of-use  assessment. 


Ease-of-Use  -  Performance  Times 

Equipment 

Setup/Removal 

(hr/vehicle) 

Data  QA/QC 
(hr/vehicle  file) 

Analysis 

(hr/event) 

#  Vehicles 

136 

136 

136 

#  Events 

14 

14 

14  ; 

Average  time  (hr) 

0.19 

0.82 

5.45 

Standard  deviation  (hr) 

0.11 

0.24 

3.57 

Standard  error  (hr) 

0.01 

0.02 

0.95  ; 
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7.0  PERFORMANCE  ASSESSMENT 

7.1  CONTROLLED  FIELD  STUDY  -  ACCURATE  VDMTS  HARDWARE 
MEASUREMENT 

The  success  criteria  assigned  to  the  position  evaluation  test  was  vehicle  positional  accuracy 
within  5.0  m  (15.4  ft)  95%  of  the  time.  The  upgraded  VDMTS  units  met  the  success  criteria.  The 
6-hr  static  positional  test  for  the  upgraded  unit  resulted  in  an  average  error  of  2.05  m  with 
99.87%  of  the  data  points  within  5.0  m  of  the  actual  location,  exceeding  the  success  criteria 
established.  A  velocity  evaluation  test  was  performed  on  the  VDMTS  system.  The  success 
criteria  assigned  to  this  test  was  vehicle  velocity  within  2.24  m/s  (5  mph)  95%  of  the  time.  The 
VDMTS  calculated  velocity  was  compared  to  the  actual  velocity.  Three  tests  were  performed 
and  the  VDMTS  was  well  within  the  criteria  at  all  times.  The  velocity  evaluation  resulted  in  a 
velocity  within  2.24  m/s  (5  mph)  of  the  true  value  for  100%  of  the  data  points. 

The  success  criteria  for  the  turning  radius  evaluation  test  was  turning  radius  within  10  m  95%  of 
the  time.  The  results  are  given  in  Table  7.  While  the  VDMTS  did  not  predict  the  turning  radius 
within  10  m  for  95%  of  the  time  at  both  velocities,  it  was  within  10  m  of  the  high  accuracy,  high 
cost  system  (the  best  prediction  available)  for  95%  of  the  test.  It  is  interesting  to  note  that  the 
unit  was  more  accurate  at  predicting  small  lower  turning  radii  (sharper  turns).  Since  vehicle 
impacts  increase  with  decreasing  turning  radii,  the  VDMTS  unit  should  allow  for  accurate 
impact  prediction.  Even  though  turning  radius  prediction  errors  increased  with  increasing  turning 
radii,  the  associated  impact  prediction  error  is  small  since  lower  impacts  are  observed  in  these 
conditions.  Although  the  VDMTS  hardware  did  not  explicitly  meet  the  success  criteria 
established,  it  performed  as  well  as  alternative  high  accuracy  systems,  indicating  metric  success. 

The  final  success  criteria  for  the  first  performance  objective  was  to  detennine  the  ability  to 
record  in  situations  when  GPS  signals  were  not  available  due  to  topography,  vegetation,  and 
related  conditions.  The  VDMTS  met  the  performance  metric  that  VDMTS  hardware  with  INS 
provides  more  accurate  dynamic  vehicle  properties  than  GPS  alone. 

7.2  CONTROLLED  FIELD  STUDY  -  ACCURATE  VDTMS  IMPACT  MODEL 
PREDICTIONS 

The  success  criteria  assigned  to  the  model  validation  phase  of  the  demonstration  for  DW  is  a 
correlation  between  predicted  and  measured  values  of  0.8  or  higher,  and  95%  of  the  predicted 
values  are  within  20  cm  of  the  actual  value.  For  vegetation  loss  (IS)  the  success  criteria  were  a 
correlation  between  predicted  and  measured  values  of  0.7  or  higher,  and  at  least  95%  of  the 
values  within  20%  vegetation  removal  accuracy.  The  criteria  set  for  RD  accuracy  were  a 
correlation  greater  than  0.6  and  95%  of  the  points  within  3.0  cm  of  the  observed  RDs.  A 
summary  of  the  results  from  the  controlled  study  at  each  location  is  given  in  Table  16.  The 
success  metric  was  generally  met  for  correlation  between  predicted  and  measured  values  for  DW 
and  IS.  For  DW,  only  50%  of  the  samples  were  within  the  20  cm  threshold.  The  theoretical  IS 
model  met  the  success  criteria  established.  While  the  RD  model  met  the  percent  samples  within 
defined  metric,  it  did  not  meet  the  correlation  between  predicted  and  measured  metric. 
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When  analyzing  the  data,  it  became  apparent  that  the  amount  of  variability  experienced  in 
measuring  impacts  was  underestimated  when  establishing  the  metrics.  In  a  validation  project, 
metrics  are  established  to  compare  results  against  some  value  to  determine  success  or  failure.  In 
this  case,  somewhat  arbitrary  metrics  had  been  established  based  on  some  previous  data 
collected.  A  better  estimation  of  the  theoretical  model  validity  may  be  comparing  results  against 
an  existing  method  of  predicting  impacts.  Prior  to  the  development  of  the  theoretical  models,  a 
statistical  regression  model  could  be  developed  for  a  specific  site/vehicle  combination  based  on  a 
field  study  similar  to  the  controlled  study.  This  empirical  model  could  be  considered  the  best 
prediction  of  impacts  given  the  variability  experienced  in  the  field. 

As  a  secondary  measure  of  theoretical  model  success,  statistical  regression  models  were 
developed  for  each  site/vehicle  combination.  It  is  important  to  note  that  these  results  are  biased 
towards  the  statistical  models  since  they  were  developed  using  the  same  dataset  used  for  the 
metric  evaluation.  The  statistical  DW  model  slightly  outperforms  the  theoretical  model; 
however,  it  is  not  close  to  meeting  the  metrics  established  (Table  16).  Theoretical  model 
perfonnance  was  very  comparable  to  the  statistical  models  (Tables  16-17). 

Table  16.  Metric  Analysis  summary  for  theoretical  and  statistical  models  in  Fort  Riley, 

Eglin  AFB,  and  PTA  Controlled  Study. 

(Note:  Units  are  cm  for  DW,  RD  and  percentage  for  IS) 


DW 

IS 

RD 

Theoretical 

Stat  Mod 

Theoretical 

Stat  Mod 

Theoretical 

Stat  Mod 

Correlation  between 
predicted  and  measured 

0.89 

0.94 

0.90 

0.90 

0.1 

0.5 

Average  error  between 
predicted  and  measured 

-0.2  cm 

2.7  cm 

-1.3% 

1.7% 

0.5  cm 

-1.4  cm 

Average  absolute  error 
between  predicted  and 
measured 

28.0  cm 

18.8  cm 

10.8% 

9.7% 

0.6  cm 

1.8  cm 

%  Samples  within  defined 
metric 

50% 

67% 

86% 

78% 

94% 

85% 

Table  17.  Average  absolute  error  between  predicted  and  measured  values  for  theoretical 

and  statistical  models  from  controlled  study. 

(Note:  Units  are  cm  for  DW,  RD  and  percentage  for  IS) 


Riley 

Eg 

lin  AFB 

PTA 

Combined 

DW 

IS 

RD 

DW 

IS 

RD 

DW 

IS 

RD 

DW 

IS 

RD 

Theoretical 

22.0 

9.8 

1.1 

33.9 

9.8 

0.1 

22.4 

13.4 

0.7 

28.0 

10.8 

0.6 

Statistical 

13.0 

11.5 

3.9 

24.2 

7.5 

0.9 

14.8 

10.8 

1.0 

18.8 

9.7 

1.8 

7.3  SINGLE  LIVE  TRAINING  EVENT  -  ACCURATE  VDMTS  HARDWARE 
MEASUREMENTS 

Single  live  training  events  were  tracked  at  Fort  Riley  and  Fort  Benning.  A  total  of  38  vehicles 
was  tracked  across  those  two  events  representing  four  vehicle  types.  The  positional  accuracy  met 
the  metric  criteria  with  an  off-road  positional  accuracy  observed  of  1.6  m  (±0.1  m).  In  several 
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instances  during  the  live  training  event  at  Fort  Riley,  the  GPS  signal  was  lost.  In  these  cases,  the 
VDMTS  INS  system  succeeded  in  calculating  vehicle  location.  To  assess  INS  perfonnance,  data 
were  analyzed  on  a  subset  of  the  vehicle  files  with  and  without  the  GPS  data  (48,259  samples). 
The  success  criteria  for  this  metric  was  vehicle  positional  accuracy  within  10  m  (32.8  ft)  for  300 
m  (984.2  ft)  after  GPS  signal  was  lost  90%  of  time.  In  the  subset  of  10  vehicles,  the  positional 
accuracy  never  exceeded  the  10  m  threshold  signifying  the  success  criteria  were  met.  The 
average  error  compared  with  the  GPS  signal  was  0.164±0.002  m.  Across  the  48,259  samples,  the 
maximum  error  compared  with  the  GPS  signal  was  2.60  m. 

7.4  SINGLE  LIVE  TRAINING  EVENT  -  ACCURATE  VDTMS  IMPACT  MODEL 
PREDICTIONS 

The  success  criteria  established  for  this  test  was  predicted  DW  within  20  cm  of  actual  DW  in 
90%  of  the  sample  sites,  predicted  vegetation  loss  within  20%  of  actual  vegetation  loss  in  80%  of 
the  sample  sites,  and  predicted  RD  within  4  cm  of  actual  RD  in  80%  of  the  sample  sites.  As 
discussed  in  Section  6.6.3,  the  predictions  from  the  statistical  regression  models  were  also 
compared  against  the  success  metrics  to  estimate  the  accuracy  of  the  theoretical  models 
compared  to  the  previous  method  of  predicting  impacts.  The  data  summarized  against  these 
metrics  are  given  in  Table  18. 

The  metrics  established  for  IS  and  RD  were  met  (IS  within  20%  and  RD  within  4  cm  of  the 
actual  values  for  at  least  80%  of  the  sampling  sites)  while  the  metric  established  for  DW  did  not 
meet  the  established  metric  (<90%  of  the  data  within  20  cm  of  actual  DW).  However,  the 
average  absolute  error  between  the  predicted  and  measured  values  for  DW  was  the  same  when 
predicted  with  the  theoretical  model  and  the  site  and  vehicle-specific  statistical  model  (29.7  cm 
versus  29.8  cm).  While  the  statistical  models  perfonned  better  than  the  theoretical  models  in  the 
controlled  study,  the  theoretical  models’  perfonnance  was  similar  to  the  statistical  models  in  the 
live  single  training  event  study.  In  fact,  the  theoretical  model  perfonned  better  than  the  site  and 
vehicle-specific  statistical  model  in  predicting  IS.  Previously,  a  separate  study  was  required  to 
develop  a  statistical  model  for  each  site  being  tested,  resulting  in  added  study  costs.  The 
theoretical  model  removes  this  necessity  while  providing  an  estimate  of  the  training  event 
impacts. 


Table  18.  Summary  of  live  training  event  tracking  at  Fort  Riley 
and  Fort  Benning  (combined). 


DW 

IS 

RD 

Theoretical 

Stat  Mod 

Theoretical 

Stat  Mod 

Theoretical 

Stat  Mod 

Average  error  between 
predicted  and  measured 

14.9  cm 

-6.1  cm 

-1.8% 

-1.7% 

0.1  cm 

-0.2  cm 

Average  absolute  error 
between  predicted  and 
measured 

29.7  cm 

29.8  cm 

6.3% 

8.3% 

0.7  cm 

0.9  cm 

%  Samples  within  defined 
metric 

45.6% 

50.6% 

95.0% 

93.1% 

98% 

100% 
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7.5  SINGLE  LIVE  TRAINING  EVENT  -  VDMTS  HARDWARE  DURABILITY 


The  last  component  of  the  single  live  training  events  was  to  assess  hardware  durability.  The 
success  metric  established  for  this  study  was  percent  of  recorded  time  >80%  of  the  actual 
military  training  time.  Hardware  durability  in  single  events  was  assessed  at  Fort  Riley,  Fort 
Benning,  and  PTA.  From  Table  1 1  in  Section  6.6.3  we  can  see  that  this  metric  was  met  at  all 
three  locations  with  a  total  percent  data  recorded  of  85.5%  across  the  three  events. 

7.6  LIVE  TRAINING  -  VDMTS  HARDWARE  DURABILITY 

The  hardware  durability  described  in  the  previous  section  was  also  assessed  across  all  training 
events  in  the  last  phase  of  the  demonstration.  For  this  assessment,  the  success  criteria  established 
was  percent  of  recorded  time  >80%  of  training  time  per  vehicle  type  for  any  event.  The 
demonstration  resulted  in  13,587.1  hr  logged  out  of  a  total  of  15,056.8  total  hr  of  training 
resulting  in  VDMTS  units  recording  90.2%  (±2.3%)  of  the  total  training  events  tracked.  In  every 
event  but  two,  >80%  of  the  data  were  collected  (77.8%  and  79.8%  on  July  13-15,  2010,  and  May 
10-17,  201 1,  respectively).  Separating  the  data  by  vehicle  type,  >80%  of  the  data  were  collected 
except  for  the  HEMTT  vehicle  (62.8±6.9%).  The  reason  for  more  data  loss  on  the  HEMTT 
vehicles  is  unclear.  This  could  be  due  to  the  mounting  location  available  for  the  HEMTT.  While 
the  failure  of  data  collection  on  the  HEMTT  is  cause  for  some  concern,  the  influence  on  the  total 
off-road  impacts  is  minor.  In  14  training  events  tracked,  very  few  HEMTTs  were  observed  going 
off-road.  A  lesson  learned  from  these  demonstrations  was  that  care  should  be  taken  to  ensure  that 
the  units  are  mounted  as  securely  as  possible,  especially  when  mounting  units  on  the  HEMTT 
vehicles. 

7.7  LIVE  TRAINING  -  QUANTITATIVE  EASE  OF  SYSTEM  USE 

The  system  ease  of  use  was  measured  quantitatively  by  determining  the  time  required  to  perform 
each  task  and  compare  with  a  time  deemed  acceptable  in  the  approved  demonstration  plan.  The 
success  criteria  were  developed  through  discussion  with  installation  staff  and  ESTCP 
management.  In  the  demonstration  plan,  the  success  metric  established  for  time  required  for 
analysis  was  40  hr  per  five  events  at  each  installation.  Since  we  were  not  able  to  track  five  events 
at  both  installations,  the  success  metric  was  modified  to  8  hr  per  event  to  account.  The  success 
metrics  and  times  required  for  each  task  are  given  in  Tables  19  and  20.  In  every  case,  the  system 
met  the  success  metrics  established  (p<0.05).  This  indicates  the  system  was  simple  enough  for 
easy  implementation  into  the  management  program  without  extensive  training  or  time 
requirements. 


Table  19.  Training  summary  for  ease  of  system  use  assessment. 


Hardware  Use 
(hr/person) 

Data  QA/QC 
(hr/person) 

Analysis 

(hr/person) 

Success  criteria  (H0) 

4.0 

4.0 

16.0  1 

Average  time  (hr) 

0.3 

1.07 

6.33 

Standard  error  (hr) 

0.23 

0.35 

2.58  1 

T-test 

(average  time  <success  criteria) 

pcO.OOOl 

pcO.OOOl 

p<0.0001 
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Table  20.  Summary  of  time  requirement  to  perform  each  step  in  the  VDMTS  process. 


Equipment  Setup/Removal 
(hr/vehicle) 

Data  QA/QC 
(hr/vehicle  file) 

Analysis 

(hr/event) 

Success  criteria  (H0) 

1.0 

1.0 

8.0 

,  Average  time  (hr) 

0.19 

0.82 

5.45 

Standard  error  (hr) 

0.01 

0.02 

0.95 

'  T-test 

'  (average  time  <success  criteria) 

pO.OOOl 

pO.OOOl 

p=0. 00958 

7.8  LIVE  TRAINING  -  QUANTITATIVE  QUALITY  AND  ACCURACY  OF  DATA 

This  performance  objective  was  designed  to  test  the  hardware  and  model  components’  ability  to 
produce  actionable  results  for  installation  land  managers.  The  first  metric  objective  was  to  assess 
the  ability  to  use  the  data  for  parameterization  of  land  management  models.  The  resolution  of 
each  specific  model  defines  the  acceptable  accuracy  of  data  for  model  parameterization.  As 
described  in  Table  1,  a  success  criteria  of  <10  m  positional  error  is  established.  Success  criteria 
for  time  off-road,  vegetation  loss,  and  IS  are  errors  less  than  20%.  The  methods  for  obtaining 
these  criteria  are  described  in  Table  1. 

The  observed  accuracy  in  this  demonstration  for  each  data  type  are  1.6  m  average  positional 
error,  1%  error  in  estimating  time  off-road  and  6.3%  error  predicting  IS.  These  measured 
accuracies  meet  the  proposed  success  criteria  for  each  data  type.  These  values  also  allow 
determination  of  the  ability  to  use  this  system  to  parameterize  different  models.  If  the  model 
requires  vehicle  tracking  data  and  uses  a  spatial  resolution  of  3  m,  the  accuracy  of  these  data  for 
model  use  is  adequate.  However,  if  an  erosion  prediction  model  required  5%  accuracy  of 
vegetation  cover  for  acceptable  results,  data  quality  from  the  VDMTS  process  (hardware  and 
models)  is  not  high  enough  for  implementation  in  the  model. 

The  second  metric  objective  was  to  determine  if  VDMTS  data  quality  is  sufficient  to  identify 
training  area  use  patterns.  Vehicle  positional  accuracy  required  for  LRAM  and  other 
maintenance  requirements  was  detennined  to  be  <10  m  positional  error.  For  TES  habitat  impact 
analysis,  a  <5  m  positional  error  is  required.  Additionally,  <20%  error  in  vegetation  removal  is 
needed  for  site  maintenance  requirements.  The  development  of  these  success  criteria  is 
documented  in  Table  1.  The  observed  accuracy  in  this  demonstration  for  each  data  type  is  1.6  m 
average  positional  error  and  6.3%  error  in  vegetation  loss  estimation.  These  measured  accuracies 
meet  the  proposed  success  criteria  for  each  data  type.  Again,  this  approach  can  be  taken  to 
determine  if  data  quality  is  acceptable  for  any  training  area  use  pattern  quantification  and 
analysis. 

7.9  LIVE  TRAINING  -  QUALITATIVE  EASE  OF  SYSTEM  USE 

In  addition  to  quantitative  ease-of-use  metrics,  a  qualitative  ease-of-use  metric  was  proposed  in 
the  demonstration  plans.  This  was  aimed  at  determining  if  the  effort  required  to  train  a  technician 
in  VDTMS  operation  and  perfonnance  of  tracking  events  and  data  analyses  was  acceptable.  The 
qualitative  metric  was  also  proposed  to  obtaining  any  feedback  from  the  installation  technicians 
on  use  of  the  systems  to  identify  any  drawbacks  to  the  system  and  learn  from  any  suggestions 
they  may  have. 
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This  metric  was  tested  by  giving  an  evaluation  fonn  to  each  technician  who  used  the  system  in 
the  demonstration.  For  each  step  of  the  data  collection,  they  were  asked  to  give  a  1-10  rating  (a 
rating  of  10  indicates  no  issues  with  that  step  while  a  rating  of  1  indicates  an  unusable  or  difficult 
step.  The  results  of  these  evaluation  forms  are  given  in  Table  21.  We  consider  an  average  rating 
>7,  indicating  no  major  issues  with  that  step  in  the  VDMTS  process. 

Table  21.  Summary  of  evaluation  forms  received  from  technicians  on  VDMTS  use. 


Task 

Average 

Rating 

Comments 

Operation  of  vehicle 
tracking  hardware 

9.3 

•  Overall,  VDM  boxes  fairly  easy  to  use 

•  Problems  w/magnetic  mounting  of  GPS  sensor 

Mounting  and  dismounting 
of  hardware 

8.7 

•  Some  issues  mounting  boxes  to  vehicles  (i.e.,  finding  secure  place 
to  mount  without  getting  in  way) 

•  Hardware  needs  to  be  placed  securely  and  oriented  correctly 

Hardware  maintenance  (e.g., 
charging/replacing  batteries, 
data  card  replacement,  etc.) 

9.5 

•  New  units  easier  to  charge  than  older  models  (pre¬ 
demonstration) — however,  would  be  nice  to  have  better  access  to 
data  card 

Downloading  data 

10 

Checking  data  on  computer 
for  errors  and  completeness 

8.5 

Processing  raw  data  fdes 

7 

•  Process  is  straightforward  and  simple  but  takes  time  to  complete  if 
many  vehicles 

Processing  data  to  determine 
vehicle  velocity  and  turning 
radius 

8 

Using  vehicle  impact 
models  for  prediction  of 
impacts 

7 

Analyzing  impact  data  for 
site  specific  summaries 

8 

Overall,  the  results  of  the  qualitative  ease  of  use  metric  indicated  that  vehicle  tracking  units  were 
easy  enough  for  the  technicians  to  work  with.  There  were  a  few  comments  and  concerns  that  can 
be  addressed  in  the  systems  in  the  future.  Since  the  VDM  units  are  a  custom  built  product,  it  is 
possible  to  request  certain  component  changes  if  requested  by  an  installation. 

7.10  LIVE  TRAINING  -  QUALITATIVE  QUALITY  AND  ACCURACY  OF  DATA 

A  similar  qualitative  metric  was  proposed  to  assess  quality  and  accuracy  of  data  for  land  use 
decision  making.  This  metric  was  proposed  to  document  issues  and  comments  installation 
technicians  had  regarding  implementing  the  VDMTS  process  in  their  programs.  Similar  to  the 
previous  section,  an  evaluation  fonn  was  given  to  each  technician  who  used  the  system  in  the 
demonstration  (See  Appendix  D  of  the  Final  Report  for  form).  Technicians  filled  in  only  the 
evaluation  sections  relating  to  their  experience  with  the  system.  A  summary  of  the  responses 
received  in  these  evaluation  forms  is  given  in  Table  22. 
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Table  22.  Summary  of  evaluation  forms  received  from  installation  land  managers  on  use  of 

data  collected  with  the  VDMTS. 


Issue 

Average 

Rating 

Comments 

Data  collected  are  of  value  for 
decision  making 

9.3 

•  This  will  be  very  useful  data  for  assessing  impacts  of  heavy 
maneuver  effects  on  Red-cockaded  Woodpeckers  (RCW) 
pertaining  to  the  Army  Reconnaissance  Course  (ARC)  when 
providing  briefings  to  the  U.S.  Fish  and  Wildlife  Service 
(USFWS)  and  our  chain  of  command. 

Maps  produced  from  data  aid 
in  visualization  and  analysis  of 
vehicle  use  patterns  and 
associated  impacts 

9 

•  This  will  be  very  useful  data  for  assessing  impacts  of  heavy 
maneuver  effects  on  RCWs  pertaining  to  the  ARC  when 
providing  briefings  to  the  USFWS  and  our  chain  of  command. 
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8.0  COST  ASSESSMENT 


8.1  COST  MODEL 

Since  there  is  not  an  existing  technology  to  compare  costs,  several  other  technology  cost 
implementation  scenarios  were  evaluated.  The  scenarios  were  intended  to  show  that  costs  vary 
between  installations  that  want  to  track  many  events  from  those  interested  in  tracking  only  a  few. 
The  scenarios  below  are  hypothetical  scenarios.  Table  25  gives  the  cost  development  approach 
for  the  scenarios. 

1 .  Scenario  1 :  An  installation  that  needs  to  track  several  events  to  answer  one  management 
question  and  then  never  use  the  technology  again.  This  scenario  is  representative  of  an 
installation  needing  to  track  military  vehicles  to  assist  with  an  environmental  impact 
statement  or  biological  assessment.  This  scenario  would  track  20  vehicles  for  5  one- 
week-long  events  over  a  period  of  1  year. 

2.  Scenario  2:  An  installation  that  needs  to  continually  track  a  certain  number  of  events 
each  year  for  an  indefinite  period  of  time.  This  scenario  is  representative  of  an 
installation  needing  to  track  military  vehicles  to  show  compliance  with  an  established 
policy  or  agreement.  This  scenario  would  track  20  vehicles  for  10  one -week-long  events 
every  year  for  5  years. 

3.  Scenario  3:  An  Army  regional  support  center  provides  support  to  all  installations 
requiring  use  of  VDMTS  services.  This  could  also  be  a  contractor  that  provides  vehicle 
tracking  support  to  installations  under  contract.  This  scenario  is  representative  of  an 
implementation  decision  to  support  multiple  installations  more  economically.  This 
scenario  is  based  on  the  U.S.  Anny  Sustainable  Range  Program  installation  support 
model.  This  scenario  would  potentially  include  10  installations  that  each  need  to  track 
20  vehicles  for  10  one-week-long  events  every  year  for  5  years. 

4.  Scenario  4:  Official  Army  weapon  and  training  system  data  (e.g.,  DFIRST,  BFT) 
become  available  to  provide  vehicle  static  and  dynamic  property  information.  For  this 
cost  analysis,  we  assumed  that  the  Army  users  (e.g.,  IT  AM  and  Environmental)  would 
have  access  to  these  datasets.  Therefore,  only  data  analysis  costs  and  possibly  some 
limited  data  acquisition  costs  are  required.  Currently  however,  this  assumption  likely 
does  not  hold  true,  and  acquisition  costs  (time  and  money)  would  be  higher  than 
assumed.  For  this  scenario,  we  used  the  same  installation  requirements  as  Scenario  2  for 
comparison  (20  vehicles  for  10  one- week-long  events  every  year  for  5  years). 

Cost  elements  estimates  are  summarized  in  Table  23.  Cost  elements  are  the  main  cost  groups 
associated  with  technology  use.  Sub-element  costs  are  a  further  breakdown  of  costs  categories 
associated  with  a  specific  cost  element.  The  intent  of  this  table  is  to  identify  all  data  and 
infonnation  that  were  tracked  through  demonstration  implementation.  These  data  were  utilized  to 
calculate  costs  for  each  Scenario  in  Table  25. 
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Table  23.  Cost  model. 


Cost 

Element 

Cost  Sub-element 

Data  Tracked 

VDMTS 

Hardware 

Cost 

Purchase 

Initial  hardware  purchase  cost  reported  as  average  cost  per  vehicle 
tracking  unit.  Initial  hardware  cost  data  collected  by  Cybernet  during 
system  production. 

Maintenance/ 

replacement 

Hardware  maintenance/replacement  cost  takes  into  account  life 
expectancy  and  maintenance  costs.  Life  expectancy  is  estimated  by 
Cybernet.  Maintenance  costs  include  part  (e.g.,  battery,  switches,  etc.) 
replacement  and  repair. 

Training 

Hardware 

Cost  of  labor  for  person  to  learn  how  to  operate  and  maintain  hardware. 

Cost  is  based  on  number  of  hours  of  training  per  person  trained  and 
average  employee  cost/hr. 

Data  processing 

Cost  of  labor  for  person  to  learn  data  processing  and  data  QA/QC 
procedures.  Cost  is  based  on  number  of  hours  of  training  per  person 
trained  and  average  employee  cost/hr. 

Data  analysis 

Cost  of  labor  for  person  to  learn  data  analysis  procedures.  Cost  is  based  on 
number  of  hours  of  training  per  person  trained  and  average  employee 
cost/hr. 

Event 

Tracking 

Preparation 

Labor  cost  to  maintain  and  prepare  systems  for  event  tracking.  Includes 
battery  recharging,  minor  repairs,  and  system  performance  checks.  Cost 
based  on  average  time  per  tracking  unit  and  average  employee  cost/hr. 

VDMTS  setup 

Labor  cost  to  install  systems  in  the  field.  Cost  based  on  average  time  per 
tracking  unit  and  average  employee  cost/hr. 

VDMTS  removal 

Labor  cost  to  remove  systems  in  the  field.  Cost  based  on  average  time  per 
tracking  unit  and  average  employee  cost/hr. 

Data  processing 

Labor  cost  to  download,  perform  quality  control,  and  preprocess  data.  Cost 
based  on  average  labor  time  for  QA/QC  preprocessing  per  tracking  unit 
per  event  multiplied  by  average  labor  cost 

Travel 

Travel  cost  associated  with  event  tracking.  This  cost  includes  airfare, 
rental  car,  and  per  diem.  This  cost  is  estimated  since  travel  costs  vary 
depending  on  distance  and  location.  This  cost  is  necessary  to  accurately 
assess  the  different  implementation  scenarios. 

Event 

Analysis 

Basic  summarization 

Cost  to  perform  basic  analysis  of  vehicle  tracking  data.  This  includes 
performing  impact  assessment  analysis  of  the  vehicle  tracking  data  with 
the  theoretical  models,  incorporating  data  into  geographic  information 
system  environment,  and  performing  basic  event  summaries.  Cost  based 
on  average  labor  time  for  basic  vehicle  impact  summarization. 

Site/question-specific 

summary 

Data  interpretation/ summarization  to  meet  land  management  objective. 

This  could  be  the  percent  of  vegetation  removal  by  location,  percent  time 
off-road,  or  amount  of  time  vehicles  spent  in  TES  habitat.  Cost  based  on 
average  labor  time  for  data  interpretation/presentation  for  land 
management  problem.  Cost  is  the  total  personnel  labor  multiplied  by  the 
average  labor  cost. 

8.2  COST  DRIVERS 

The  cost  drivers  for  implementing  the  VDMTS  process  and  detennining  which  application  and 
technology  is  most  cost  effective  are  highly  dependent  on  the  specific  situation  at  each 
installation  and  the  issues  being  addressed.  For  example,  it  is  anticipated  that  larger  installations 
with  more  personnel  would  be  able  to  implement  the  VDMTS  process  in-house.  In  contrast,  a 
small  branch  (only  a  few  people)  at  a  smaller  installation  may  need  to  hire  a  regional  support 
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center  to  perform  the  analyses  needed.  Additionally,  the  cost  for  implementing  the  system 
depends  on  the  issues  being  addressed  and  the  data  needed  to  answer  those  questions.  An 
installation  needing  to  track  only  a  few  events  could  hire  a  regional  support  center  to  perform  the 
analysis  required.  The  following  section  expands  upon  these  anticipated  cost  drivers  and 
provides  recommendations  for  system  implementation  given  different  scenarios. 

8.3  COST  ANALYSIS  AND  COMPARISON 

8.3.1  Monitoring  Methods  and  Costs 

A  cost  analysis  was  perfonned  for  the  scenarios  defined  in  Section  8.1  utilizing  the  data  collected 
as  described  in  Table  23.  The  cost  data  described  in  Table  23  were  collected  throughout  the 
demonstration  from  a  number  of  sources.  Hardware  costs  were  obtained  from  the  suppliers 
(Cybernet,  parts  suppliers,  etc.).  Hardware  maintenance  costs  were  calculated  from  the  costs 
observed  throughout  the  demonstration.  Time  requirements  for  each  component  were  obtained 
through  this  demonstration  (Perfonnance  Objective  7).  The  actual  life  span  depends  on  the 
severity  of  the  conditions  the  unit  is  being  used  in,  the  average  length  of  deployment,  and  other 
factors.  Because  the  cost  of  implementation  is  highly  dependent  on  the  situation,  an  extensive 
life-cycle  cost  analysis  is  difficult  without  making  a  number  of  assumptions.  In  order  to  estimate 
life-cycle  costs  and  yearly  costs,  the  cost  analysis  approach  taken  is  to  assume  the  different 
scenarios  described  in  Section  8.1.  A  life-cycle  cost  analysis  of  the  hardware  was  performed  in 
order  to  estimate  a  maintenance/replacement  cost  per  event  (Table  24). 

Table  24.  Life-cycle  cost  analysis  of  VDMTS  hardware. 


Life-cycle  Cost  Component 

Cost/Unit 

Replacements 

Required/Life 

Span 

Cost/Life 

Span 

Initial  purchase  cost 

$2900.00 

1 

$2900.00 

Battery  replacement 

$36.86 

2 

$73.72 

GPS  antenna  replacement 

$31.00 

1 

$31.00 

Switch  replacement 

$7.00 

2 

$14.00 

Wiring  and  connections 

$20.00 

2 

$40.00 

Misc.  costs  (data  cards,  card  connectors,  etc.) 

$30.00 

4 

$120.00 

Total  replacement  cost 

$3178.72 

Estimated  unit  life  span 
(#  events) 

100 

Maintenance-Replacement 
Cost  per  Event  per  Unit 

$31.79 
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Table  25.  Cost  model  for  alternative  fielding  scenarios. 


Cost 

Element 

Cost  Sub-Element 

Costing  Analysis  Scenario 

Installation 
Performed 
(Scenario  1  and  2) 

Regional  Support 
Center 
(Scenario  3) 

Army  Standard 
System 

(DFIRST/BFT) 
(Scenario  4) 

Hardware 

Purchase 

$2900/unit 

$2900/unit 

NA 

Maintenance/replacement 

$32/unit/event 

$32/unit/event 

NA 

Training 

Hardware 

4  hr/class@$37/hr 

4  hr/class@$37/hr 

NA 

Data  processing 

4  hr/class@$37/hr 

4  hr/class@$37/hr 

4  hr/class@$37/hr 

Data  analysis 

16  hr/class@$53/hr 

16  hr/class@$53/hr 

16  hr/class@$53/hr 

Event 

Tracking 

Preparation 

0.5  hr/unit@$37/hr 

0.5  hr/unit@$37/hr 

0.0  hr/unit 

Setup 

0.3  hr/unit@$37/hr 

0.3  hr/unit@$37/hr 

0.0  hr/unit 

Removal 

0.1  hr/unit@$37/hr 

0.1  hr/unit@$37/hr 

0.0  hr/unit 

Data  processing 

1.0  hr/unit@$37/hr 

1.0  hr/unit@$37/hr 

1.0  hr/unit@$37/hr 

Travel 

$0/event 

$  1500/event 

$0/event 

Event 

Analysis 

Basic  summary 

1.0  hr/vehicle@$53/hr 

1.0  hr/vehicle@$53/hr 

1.0  hr/vehicle@$53/hr 

Site  specific  summary 

8.0 

hr/summary  @$5  3  /hr 

8.0 

hr/summary  @$5  3  /hr 

8 .  Ohr/summary  @$5  3  /hr 

Total 

Costs 

Purchase  costs 

$2900/unit 

$2900/unit 

$0  i 

Training  costs/individual 

$1 144/individual 

$1 144/individual 

$996/individual 

Fixed  cost/  event 

$499/event 

$  1999/event 

$424/event 

Cost/vehicle  tracked/event 

$123/vehicle 

$123/vehicle 

$90/vehicle 

Cost  for  Proposed 
Scenarios 

Scenario  1: 

$76,954 

($76,954/year/site) 
($15,391  event) 

Scenario  2: 

$237,244 

($47,449/year/site) 

($4, 745/event) 

$2,649,288 
($53, 185/year/site) 

($5, 299/event) 

$112,196 

($22, 439/year/site) 

($2, 244/event) 

As  illustrated  in  Figure  21,  a  regional  support  center  does  not  actually  reduce  total  costs  to  the 
Anny  using  the  given  costs.  This  is  due  to  the  travel  costs  required  for  technicians  to  travel  to  the 
site.  The  cost  of  $1500/event  (based  on  airfare,  hotel,  and  per  diem  for  the  week)  outweighs  any 
benefit  the  regional  support  center  gives  from  a  reduction  in  equipment  and  labor  needed  per 
installation.  If  these  travel  costs  are  lower  (e.g.,  due  to  proximity  to  installation),  the  slope  of  the 
Regional  Support  Center  line  in  Figure  21  is  reduced  and  it  can  become  less  expensive  for  a 
regional  support  center  to  perform  the  studies.  This  cost  analysis  is  limited  by  the  data  collected 
through  this  demonstration.  Since  this  demonstration  estimated  only  a  cost  requirement  to 
perfonn  the  VDMTS  process,  the  relationship  between  cost  and  number  of  events  (or 
installations)  is  a  linear  function.  This  assumption  is  accurate  in  the  case  of  Scenario  1  and  2.  As 
more  installations  adopt  the  VDMTS  method  under  these  scenarios,  the  total  cost  to  the  military 
increases  linearly  as  the  efforts  are  replicated  for  each  installation.  In  the  case  of  Scenario  3,  as 
the  number  of  events  tracked  increased,  the  cost  per  event  would  decrease  due  to  economies  of 
scale  (increased  efficiency  of  labor,  decreased  analyses  times,  etc.).  However,  the  cost  data 
collected  in  this  demonstration  do  not  allow  for  the  detennination  of  the  decreasing  costs  per 
event  collected  for  Scenario  3. 
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Number  Installations 


Figure  21.  Total  cost  to  Army  from  different  scenarios  with  increasing  number  of 

installations  adopting  process. 

(Assumes  10  events  of  20  vehicles/event  at  each  installation  over 
5  years  at  the  costs  given  in  Table  25) 

8.3.2  Alternative  Monitoring  Methods  and  Costs 

Alternative  options  to  the  VDMTS  system  were  also  identified  through  discussion  with  military 
land  managers.  However,  the  VDMTS  process  provides  land  managers  with  data  and  products 
that  cannot  be  produced  with  either  of  these  monitoring  methods.  This  cost  comparison  can  give 
an  idea  of  VDMTS  implementation  costs  compared  with  existing  monitoring  costs.  Fort  Riley  is 
currently  implementing  a  land  change  detection  model  based  on  bimonthly  250  m  resolution 
Moderate  Resolution  Imaging  Spectroradiometer  (MODIS)  data  at  an  estimated  operating  cost  of 
$2418.13  per  year.  However,  this  product  was  recently  developed  and  is  currently  available  only 
at  Fort  Riley.  As  such,  accurate  cost  data  for  implementation  at  other  installations  are  not 
available  for  comparison  to  the  VDMTS  process.  The  second  alternative  option  is  the 
“windshield  survey,”  which  is  used  to  determine  locations  of  highly  impacted  training  areas  and 
to  locate  LRAM  sites.  Fort  Riley  currently  employs  this  method  at  an  estimated  cost  of 
$47, 449/year/installation.  While  VDMTS  implementation  does  not  completely  eliminate  the 
need  for  field  validation  of  these  sites,  tracking  a  number  of  events  per  year  could  reduce  this 
cost  and  free  technician  time  to  repair  more  of  these  impacted  sites.  Another  added  benefit  which 
is  not  quantified  in  this  analysis  is  the  ability  to  use  the  VDMTS  data  for  NEPA  and  ATTACC 
reporting  and  assessments. 
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9.0 


IMPLEMENTATION  ISSUES 


9.1  VDMTS  ACCEPTANCE,  ISSUES,  AND  ALTERNATIVES 

Once  individuals  used  the  VDMTS  units  and  utilized  the  data  they  collected,  they  were  often 
impressed  and  excited  to  implement  them  into  their  programs.  Initially,  the  technicians  were 
generally  hesitant  to  commit  too  much  of  their  time  to  using  the  systems.  After  using  the  systems 
once,  they  were  often  surprised  by  the  lack  of  time  and  effort  required  to  collect  the  data.  At  Fort 
Riley  and  PTA,  the  use  of  the  VDMTS  system  was  implemented  into  the  installation  proposed 
IT  AM  5-year  plans.  Staff  at  a  third  installation  requested  a  proposal  to  track  additional  training 
events  after  the  ESTCP  demonstration  project  to  support  an  existing  research  program  on  base. 
However  despite  the  installation  acceptance  of  the  process,  turnover  at  ITAM  and  Environmental 
installation  branches  may  result  in  VDMTS  system  implementation  issues.  Additionally,  there 
seems  to  be  little  continuity  in  programs  as  this  turnover  occurs.  An  original  installation  staff 
member  may  find  the  VDMTS  collected  data  and  analysis  invaluable  towards  their  program; 
however,  their  replacement  may  not  have  the  time  or  desire  to  leam  how  the  system  could 
support  their  work. 

Currently,  the  VDM  tracking  units  are  custom  built  by  Cybernet.  A  lower  cost  alternative 
(Vehicle  Tracking  System  [VTS]  unit)  without  the  INS  tracking  capability  can  be  built  using 
standard  commercial  off-the-shelf  (COTS)  components.  User  manuals  for  the  VDM  units  are 
supplied  in  Appendix  C  of  the  Final  Report.  The  factors  involved  with  implementing  either 
system  or  hiring  a  regional  support  center  to  collect  tracking  data  are  summarized  in  the  Cost 
Model  section  (Section  8.1).  The  main  driver  for  these  decisions  is  the  availability  of  in-house 
labor  and  capability  to  perfonn  simple  analysis  of  the  vehicle  tracking  data. 

An  additional  option  for  implementation  involves  using  existing  military  standard  systems 
(Army’s  BFT  and  National  Guard’s  DFIRST),  which  obtain  vehicle  location  and  time  data  on 
live  training  events  for  post-event  analysis.  While  this  option  reduces  the  labor  required  for  the 
collection  of  vehicle  positional  infonnation,  it  presents  a  whole  new  set  of  implementation 
issues.  The  primary  issue  involves  getting  permission  to  use  these  data.  Some  of  these  data  are 
classified  and  would  need  to  be  declassified  prior  to  obtaining  them.  Secondly,  the  quality  of  the 
data  (positional  and  temporal)  may  not  match  those  validated  under  this  demonstration  plan.  A 
study  is  currently  being  performed  to  investigate  data  quality  from  these  systems  compared  with 
the  data  collected  from  the  VDM  and  VTS  systems. 

9.2  TECHNOLOGY  TRANSFER  AND  IMPLEMENTATION 

The  demonstration  plans  outlined  multiple  methods  of  tech  transfer  and  implementation  to 
improve  military  land  management  decision  making.  However,  as  mentioned  in  the  previous 
sections,  our  installation  hosts  often  found  value  to  the  data  collected  through  this  work  in  ways 
we  had  not  anticipated.  The  following  section  describes  some  of  the  different  ways  the  data 
collected  through  this  project  have  been  used  outside  the  scope  of  this  project. 

Fort  Benning  found  value  in  characterization  of  vehicle  travel  and  training  area  use.  Discussion 
with  installation  staff  led  to  the  concept  of  analyzing  vehicle  tracking  data  to  detennine  distances 
to  RCW  habitat.  Previous  work  determined  flushing  responses  to  military  training  at  different 
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distances.  These  data  allowed  Fort  Benning  to  begin  to  estimate  how  much  a  certain  training 
event  could  affect  populations.  Monitoring  the  vehicles  with  VDM  tracking  units  was 
instrumental  in  gaining  the  approval  of  the  USFWS  to  allow  increased  vehicular  traffic  around 
RCW  clusters  that  were  due  to  changes  to  the  Program  of  Instruction  (POI)  for  the  ARC. 
Coordination  with  Fort  Benning  is  ongoing  to  augment  an  existing  research  project  aimed  at 
investigating  heavy  maneuver  effects  on  RCWs.  These  data  help  quantify  and  characterize  the 
extent  of  vehicular  training  in  RCW  habitat. 

Through  this  project  we  were  able  to  support  an  Armor  School  Command  sponsored  Good  Hope 
Soil  Disturbance  Demo  project  at  Fort  Benning.  This  demonstration  informed  command  on 
expected  soil  disturbance  from  training  events  in  the  newly  constructed  training  areas.  We  were 
able  to  support  this  study  by  supplying  quantitative  data  on  impacts  and  vehicle  maneuvers. 

Coordination  is  ongoing  with  ESTCP  project  RC -200820  to  ensure  that  the  data  we  collected  at 
Fort  Riley  and  PTA  can  be  used  as  an  input  to  the  Kinematic  Wave  Rapid  Soil  Erosion 
Assessment  Model.  Vehicle  tracking  data  (vegetation  removal  and  RD)  collected  at  PTA  have 
been  used  in  the  model  to  improve  estimation  of  vegetation  cover  maps  (bare  versus  vegetation). 
These  data  can  also  improve  the  Digital  Elevation  Model  (DEM)  data  as  a  vehicle  rut  can 
concentrate  overland  flow  and  increase  the  probability  for  gully  formation. 

Data  collected  from  this  project  are  currently  infonning  the  model  development  for  an  ERDC  6.2 
study  Optimal  Allocation  of  Land  for  Training  and  Non-Training  Uses  (OPAL).  The  objective  of 
the  OPAL  work  package  is  to  predict  impacts  for  cumulative  military  land  use  activities  and 
provide  optimization  routines  for  military  land  managers.  This  informs  land  management 
decisions  and  allows  for  estimation  of  past,  present,  and  future  impacts  given  historical  and 
planned  land  use.  All  the  separate  training  events  we  tracked  through  this  study  are  being 
combined  into  a  single  database.  Impacts  and  training  distribution  will  be  characterized  be 
mission  type  allowing  more  accurate  predictions  of  impacts  from  planned  training  events. 

The  theoretical  models  are  also  being  adapted  and  modified  for  improvement  of  vehicle  mobility 
and  power  requirement  models.  A  U.S.  Anny  Tank  Automotive  Research,  Development  and 
Engineering  Center  (TARDEC)  funded  project,  Advanced  Vehicle/Terrain  Interaction  Modeling 
to  Support  Power  and  Energy  Analysis,  has  incorporated  the  DW  models  as  well  as  the  soils 
database  generated  through  this  demonstration’s  field  studies. 

As  described  in  this  section,  this  project  was  successful  in  going  outside  the  initial  project  scope 
by  providing  interested  parties  data  and  summaries  for  improved  understanding  of  mission 
impacts  to  soil  and  vegetation.  This  is  partially  due  to  the  variety  of  backgrounds  of  personnel 
who  were  involved  with  this  project.  Data  and  summaries  from  this  project  were  used  to  brief  the 
Headquarters,  Department  of  the  Army  (HQDA)  IT  AM  program  manager,  USFWS  regulators  at 
Fort  Benning,  and  training  commands.  People  from  varying  backgrounds  understood  how  the 
data  collected  could  be  used  to  inform  their  land  use  and  training  decisions  and  model 
development.  Implementation  of  this  technology  requires  consideration  of  the  infonnation 
desired  and  the  different  options  for  data  collection.  Decisions  to  implement  the  VDMTS  process 
must  take  into  account  the  question  being  asked  and  resources  available  to  the  installation. 
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